IIT Madras Generative AI Lectures: Mathematical Foundations of Generative AI
BSDA500273 lectures12 weeks
Weeks
- Week 18 lecturesCourse outline deep generative models · Introduction & problem setting | generative AI basics explained · F-divergence | variational divergence minimization in generative models · Variational divergence minimization · Tutorial 1: Forward pass & backpropagation · Tutorial 2: Introduction to pytorch: tensors · Tutorial 3: Introduction to pytorch: datasets & dataloaders · Tutorial 4: Introduction to pytorch: model building
- Week 24 lecturesGenerative modelling via variational divergence minimization · Generative adversarial networks: introduction · Generative adversarial networks: formulation · Tutorial: Tutorial: implementation of generative adversarial network
- Week 33 lecturesGANs as classifier-guided generative sampler · Deep Convolution GANs and Conditional GANs · Tutorial: Implementation of DC-GAN
- Week 410 lecturesSaturation of GAN training · Wasserstein GANs · Inversion with GANs · Bi-directional GANs · GAN inversion via latent regression · Domain Adversarial Networks · Evaluation of Generative Models · Tutorial: Implementation of Bi-GAN · Tutorial: Implementation of UDA · Tutorial: Implementation of WGAN
- Week 56 lecturesIntroduction to latent variable models · Evidence Lower Bound (ELBO) · Gaussian Mixture Models: Expectation-Maximization Algorithm · Variational Autoencoder (VAE) · Proof of Jensen's inequality · GMM
- Week 67 lecturesTraining VAE: Reparameterization methods · Training VAE · Inference with a trained VAE · Beta-VAE · Vector Quantized VAE (VQ-VAE) · Implementation of VAE · Implementation of VQ-VAE
- Week 73 lecturesDenoising Diffusion Probabilistic Models (DDPMs) · DDPM: Formulation · U-Net
- Week 88 lecturesELBO for DDPM : Part 1 · ELBO for DDPM : Part 2 · Optimization of DDPM loss · ELBO Equivalence · Training of DDPM · Inference in DDPM · Implementation of DDPM · Proofs
- Week 99 lecturesalternate interpretations of DDPMs · DDPMs as score-predictors · Guided Difusion Models · Latent Diffusion Models · Denoising Difusion Implicit Models (DDIMs) · Inference in DDIM · Implementation of DDPM Noise estimation · Implementation of DDIM · Implementation of Guided DDPM
- Week 107 lecturesAuto-Regressive Models · Attention Mechanism · Transformers for Auto-Regressive Models · Transformers architecture · Transformers: Skip Connections and Normalization · Transformers: Position Embeddings · Transformers: Training and Inference
- Week 115 lecturesAn overview of Reinforcement Learning · Policy Gradient Theorem · Expressing an AR-LM as RL policy · Proximal Policy Optimization (PPO) · Trust Region Policy Optimization (TRPO)
- Week 123 lecturesReward-Modelling · Direct Preference Optimization (DPO) · State-space-Models