Events

CILVR Seminar: Everything in AI works because it's Variational Inference

Speaker: Stephan Mandt (UC Irvine)

Location: 60 Fifth Avenue, Room 7th Floor Open Space
Videoconference link: https://nyu.zoom.us/j/98954373715

Date: Wednesday, September 23, 2026

Abstract: The mid-2010s saw a wave of deep latent-variable models derived from probabilistic and information-theoretic principles. A decade later, this perspective has fallen out of fashion as attention has shifted to autoregressive transformers, diffusion models, and their relatives. Yet variational principles are more than mathematical justification to be tucked into an appendix: they provide a starting point for designing new models and algorithms. 
 
For diffusion models, variational inference extends beyond training: a pretrained model can also serve as a prior for a new, generally intractable inference problem at test time. We incorporate new evidence through variational posterior approximations of varying expressiveness, balancing approximation quality against computational cost. Within this framework, we formulate diffusion guidance as variational control, derive principled algorithms for inverse problems, and develop new fine-tuning objectives for diffusion models. In language modeling, we separate noise injection from prediction to construct normalizing-flow-inspired latent-variable models. Shared latent randomness coordinates joint token prediction, enabling parallel generation while preserving dependencies between tokens. Thus, variational inference offers an exceptionally flexible lingua franca for generative modeling, with new training objectives following directly from textbook principles.
 
Bio: Stephan Mandt is a Professor of Computer Science and Statistics at the University of California, Irvine, Director of the UCI Center for Machine Learning, and co-director of UCI's AI in Science Institute. His research contributes to the foundations and applications of generative AI, with a focus on diffusion models, uncertainty quantification, neural compression, resource-efficient inference, and AI-driven scientific discovery. He is a Chan Zuckerberg Investigator, NSF CAREER awardee, Kavli Fellow, and former AISTATS Program and General Chair.