Variational Autoencoder (VAE)

A generative neural network that learns to compress data into a probabilistic latent space, then sample from it to generate new examples.

A Variational Autoencoder (VAE) is a type of generative model introduced by Kingma and Welling in 2013. Like a regular autoencoder, it has an encoder that compresses input data into a smaller "latent" representation and a decoder that reconstructs the input from that representation. The twist: instead of encoding to a fixed vector, the VAE encodes to a probability distribution (typically a Gaussian with a learned mean and variance), then samples from it before decoding. This probabilistic structure matters because it forces the latent space to be smooth and continuous. You can sample any point from the distribution and decode it into a plausible new example — making VAEs genuinely generative, not just compressive. The training objective combines a reconstruction loss with a KL-divergence term that keeps the latent distribution close to a standard normal. A common analogy: a regular autoencoder memorizes specific photos as exact coordinates on a map; a VAE instead learns "regions" on the map where similar photos live, so picking a random point in a region still gives you a sensible photo. VAEs were widely used for image generation (faces, digits), molecular design, and anomaly detection before diffusion models took over much of the generative-image space. They still appear inside larger systems — for example, the VAE inside Stable Diffusion compresses images into the latent space where the diffusion process actually runs. Related concepts to explore next: autoencoder, latent space, GAN, diffusion model, KL divergence, latent diffusion.