Events

CS Colloquium: Foundations of Deep Learning: Optimization and Representation Learning

Speaker: Alexandru Damian, Princeton University

Location: 60 Fifth Avenue, Room 150

Date: Tuesday, March 4, 2025

Deep learning's success stems from the ability of neural networks to automatically discover meaningful representations from raw data. In this talk, I will describe some recent insights into how optimization enables this learning process. First, I will explore how gradient descent enables neural networks to adapt to low-dimensional structure in the data, and how these ideas naturally extend to understanding the emergence of in-context learning in transformers. I will then discuss my work toward a predictive theory of deep learning optimization that characterizes how different optimizers navigate deep learning loss landscapes and how these different behaviors affect training efficiency, stability, and generalization.