Events

CILVR Seminar: How Much Model Capacity is Needed for Reasoning? A Scaling Law for Implicit Reasoning in Language Models

Speaker: Xinyi Wang

Location: 60 Fifth Avenue, Room 7th Floor Open Space
Videoconference link: https://nyu.zoom.us/j/95180893456

Date: Wednesday, March 11, 2026

Large language models exhibit impressive reasoning abilities, yet it remains unclear how much model capacity is actually required for reasoning to emerge during pretraining. In this talk, I study the minimal parameter budget required for implicit reasoning—the ability to infer new facts from learned knowledge without explicit chain-of-thought supervision. To isolate this phenomenon, we pretrain language models from scratch on synthetic corpora derived from knowledge graphs, where reasoning corresponds to multi-hop inference over graph structures. From both a theoretical and an empirical perspective, we identify a scaling law linking this optimal parameter budget to a graph search entropy measure.

Bio: Xinyi Wang is a Postdoctoral Researcher at the Princeton Language and Intelligence Lab, working closely with Danqi Chen. She obtained her Ph.D. at the University of California, Santa Barbara (UCSB), where she was advised by William Yang Wang. Her research focuses on developing a principled understanding of large foundation models from their pretraining data distribution, with the goal of improving their capabilities, addressing their limitations, and optimizing their application across diverse domains.