Events

Global AI Frontier Lab Seminar Series: Multi-Task Bayesian In-Context Learning

Speaker: Qingyang Zhu

Location: 1 MetroTech Center, Room Global AI Frontier Lab (Floor 22)

Date: Monday, June 1, 2026

Dinner & networking will begin at 6:00 PM and the seminar will start at 7:00 PM EST. In-person attendance is strongly encouraged for Lab researchers in NYC. All attendees must RSVP to participate. For online attendees, a Zoom link will be sent out prior to the event. Please reach out to global-ai-frontier-lab@nyu.edu with any questions.

Abstract: Bayesian predictive inference provides a principled framework for uncertainty quantification, data efficiency, and robust generalization. However, exact inference is often intractable, and scalable approximations may remain computationally expensive or require restrictive modeling assumptions that degrade predictive performance. Prior-Data Fitted and in-context learning networks have recently emerged as an amortized alternative by learning to map datasets directly to predictive distributions, but existing approaches are tightly coupled to the support of the training prior and lack explicit mechanisms for adapting to new priors at test time, resulting in limited robustness under distribution shift. We introduce a multi-task in-context learning framework for amortized hierarchical Bayesian predictive inference that explicitly represents prior information as a prefix of in-context datasets. A transformer trained on sequences of prior and target tasks learns to adapt its predictions across families of priors. On a suite of evaluations with increasing difficulty, including out-of-meta-distribution heavy-tailed priors and priors with high-dimensional latent structures, our method matches oracle Bayesian predictors while being orders of magnitude faster.  

 

Bio: Qingyang is a second-year PhD student at the NYU Center for Data Science, co-advised by Eric Oermann and Kyunghyun Cho. She is interested in understanding the mechanisms and inductive biases underlying robust out-of-distribution generalization and reasoning in neural networks. Previously, she received her Bachelor’s degree in Computer Science from ShanghaiTech University, where she worked on compositional syntactic language models.