October 23, 2026 (10:00 – 11:30 am)
Gross Hall 270
Arthur Spirling
Class of 1987 Professor of Politics
Director, Center for Statistics and Machine Learning
Princeton University
What to Do When Humans Are No Longer the Gold Standard: Large Language Models, Robustness and Aggregation
Large language models (LLMs) are increasingly capable of performing data annotation and other research tasks at, and in some cases beyond, the level of highly trained human experts. What does this mean for political science research and the conclusions we draw from it?
In this talk, Arthur Spirling will examine how increasingly capable LLMs may affect inference in downstream research. He will discuss why estimated treatment effects may become larger as models improve, while apparent null effects may become more difficult to interpret. The talk will also consider the importance of testing the sensitivity and robustness of research findings to future advances in AI, including the use of local calibration.
Spirling will also address the debate over proprietary “state-of-the-art” models versus open-weight, replicable alternatives, and explore when combining multiple LLMs can improve research inferences. He will discuss key identification requirements and introduce a dependence-aware approach for combining models.
