Events
MaD Seminar: Learning single index model with gradient descent: spectral initialization and precise asymptotics
Speaker: Yandi Shen
Location: 60 Fifth Avenue, Room 150
Date: Thursday, November 6, 2025
Non-convex optimization plays a central role in many statistics and machine learning problems. Despite the landscape irregularities for general non-convex functions, some recent work showed that for many learning problems with random data and large enough sample size, there exists a region around the true signal with benign landscape. Motivated by this observation, a widely used strategy is a two-stage algorithm, where we first apply a spectral initialization to plunge into the region, and then run gradient descent for further refinement. While this two-stage algorithm has been extensively analyzed for many non-convex problems, the precise distributional property of both its transient and long-time behavior remains to be understood. In this work, we study this two-stage algorithm in the context of single index models under the proportional asymptotics regime. We derive a set of dynamical mean field equations, which describe the precise behavior of the trajectory of spectral initialized gradient descent in the large system limit. We further show that when the spectral initialization successfully lands in a region of benign landscape, the above equation system is asymptotically time translation invariant and exponential converging, and thus admits a set of long-time fixed points that represents the mean field characterization of the limiting point of the gradient descent dynamic. As a proof of concept, we demonstrate our general theory in the example of regularized Wirtinger flow for phase retrieval. Based on joint work with Yuchen Chen.
Bio: Yandi Shen is currently an assistant professor in the Department of Statistics and Data Science at Carnegie Mellon University. He obtained his PhD in Statistics from University of Washington in 2021, and then spent two years at University of Chicago as a Kruskal instructor and one year at Yale University as a postdoctoral researcher. He is broadly interested in nonparametric and semiparametric statistics, high dimensional inference, and applied probability.