Events

Transformers for Maths, and Maths for Transformers

Speaker: Francois Charton

Location: 60 Fifth Avenue, Room 7th floor common area

Date: Thursday, September 21, 2023

Transformers can be trained to solve problems of mathematics and theoretical physics. I present recent work on predicting integer sequences using symbolic regression, and investigating the scattering amplitudes of gluons. On the other hand, problems of mathematics provide a benchmark for understanding transformers. Using two examples from linear algebra and integer arithmetic, I show that model predictions can be explained, that trained models do not confabulate, and that carefully choosing the training distributions can help achieve better, and more robust, performance.