Events

On Implicit Bias and Provable Generalization in Overparameterized Neural Networks

Speaker: Gal Vardi

Location: 60 Fifth Avenue, Room 650

Date: Wednesday, September 27, 2023

When training large neural networks, there are typically many solutions that perfectly fit the training data. Nevertheless, gradient-based methods have a tendency to reach those which generalize well, and understanding this "implicit bias" has been a subject of extensive research. Surprisingly, trained networks often generalize well even when perfectly fitting noisy training data (i.e., data with label noise), a phenomenon called “benign overfitting”. In this talk, I will discuss these phenomena in two-layer neural networks. First, I will show how the implicit bias can lead to solutions that generalize well, but are highly vulnerable to adversarial examples. Then, I will discuss in what settings benign overfitting occurs.