Teaching myself diffusion models by coding one from scratch
End goal is to be able to code something following the basic concepts of diffusion, train it, and hopefully sample something remotely resembling MNIST, given the compute I can get lol
- Dataloader for MNIST
- Sinusoidal time embedding
- Linear Noise schedule
- Simple UNet based architecture
- Forward and reverse diffusion process with unlearnt variance
- Sampling images
- Cosine noise schedule
- EMA for weights
- Class conditioned sampling
- Classifier guidance
- Classifier free guidance
- Latent Diffusion Model
- Learned variance
- Hybrid loss ( VB + Simple )
- Loss based importance sampling of time
- Actual Unet artchitecture from the paper, with attention
Able to generate good samples for MNIST