Subject
3 entries
Generative Models
Bookmarks
An Introduction to Variational Autoencoders
The canonical tutorial on Variational Autoencoders by Kingma and Welling — the original VAE inventors. Covers the ELBO, reparameterization trick, and extensions to deeper generative models. Essential background for anyone working with latent variable models or modern diffusion/flow models.
DIFFUSER: Discrete Diffusion via Edit-Based Reconstruction
Introduces DIFFUSER, an edit-based text generation model that adapts denoising diffusion to discrete text by framing generation as iterative editing rather than left-to-right token production. Competitive with autoregressive models on translation and summarization while enabling unique capabilities like prototype-conditioned generation and iterative revision.
Generative Pretraining from Pixels (iGPT)
The iGPT paper from OpenAI (ICML 2020) showing that a GPT-2-scale transformer trained to autoregressively predict pixels learns strong image representations — 96.3% accuracy on CIFAR-10 with a linear probe. It's a direct transposition of NLP pretraining ideas to the image domain, predating CLIP and DALL-E.