Events

CDS Seminar: Using cognitive science to explore the symbolic limits of large language models

Speaker: Tom Griffiths

Location: 60 Fifth Avenue, Room 150

Date: Friday, March 14, 2025

Large language models are an impressive demonstration of how training a neural network on symbolic data can result in what looks like symbolic behavior. However, the limits of this symbolic behavior reveal some of the ways in which the underlying model and training regime can have unexpected consequences. I will highlight four such cases: ambiguous representations of numbers, influence of prior distributions on solutions to deterministic problems (“embers of autoregression”), paradoxical effects of chain-of-thought prompting, and implicit associations revealed through behavioral prompts. Each case makes use of specific tools from cognitive science — rational analysis, similarity judgments, and experimental methods designed to evaluate the impact of verbal thinking on behavior and reveal implicit biases.