Stochastic Parrots (Bender et al., 2021)
Influential 2021 paper arguing that large language models combine linguistic forms according to statistical patterns without grounding in meaning, and cataloguing the social and environmental risks of scaling them.
"On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" was published at the 2021 ACM FAccT conference by Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell. The paper's central image describes a language model as a system that haphazardly stitches together sequences of linguistic forms it has observed in vast training data, weighted by probabilistic information about how they combine, without any reference to meaning — like a parrot reproducing speech without understanding. The argument builds on Bender's earlier work with Alexander Koller in Climbing towards NLU, which distinguished linguistic form (the observable units of language) from meaning (the mapping from language to extralinguistic referents). Because training data consists only of form, the authors argue that scaling alone cannot deliver understanding. The paper identifies several concrete risks: environmental costs from training large models, inscrutable training corpora that encode demographic and political biases, opportunity costs that crowd out other research directions, and the confabulation risk that fluent generated text will be mistaken for trustworthy information. Its publication was entangled with Gebru's and Mitchell's departures from Google and became a reference point in subsequent debates over LLM safety, evaluation, and deployment.