Reasoning vs Memorization in LLMs
When a language model solves a math problem or logic puzzle, it is often impossible to tell from the output alone whether it actually reasoned or recalled a near-duplicate from training. The distinction matters because memorization-driven scores do not generalize. Diagnostic tests focus on variant perturbations, novel composites, and how performance scales with chain-of-thought length.
The same surface behavior — a correct final answer — can be produced by two very different processes inside an LLM: pattern matching against a near-duplicate seen during pretraining, or step-by-step inference over the specific premises in the prompt. From the output alone, these are hard to distinguish, which is why headline scores on popular benchmarks like GSM8K or MMLU can overstate what a model actually does on a new instance. The reason it matters: memorization-driven accuracy does not transfer to even small variations of the same task, while genuine reasoning should. Indicators of memorization. The clearest signal is performance that collapses under cheap rewrites that leave the underlying problem intact. Razeghi et al. (2022), "Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning," showed that few-shot arithmetic accuracy correlates strongly with how often the specific operands appeared together in the pretraining corpus — evidence that models extrapolate less than the benchmark implies. Apple's GSM-Symbolic Benchmark (Mirzadeh et al., 2024) generates symbolic templates of GSM8K problems and shows that simply renaming variables or swapping numbers degrades accuracy across all frontier models, and that inserting a single irrelevant clause can drop accuracy by up to 65% — behavior consistent with template matching rather than reading the problem. Indicators of reasoning. The opposite signature is performance that holds across surface variants, scales with longer Chain-of-Thought Prompting: How Step-by-Step Reasoning Improves LLM Accuracy, and produces intermediate steps that a checker can verify independently. Wei et al. (2022) established that chain-of-thought gains emerge only at sufficient scale, suggesting that pure recall does not benefit from extra reasoning tokens but stepwise inference does. Saparov & He (2022), "Language Models Are Greedy Reasoners," introduced the PrOntoQA synthetic deduction dataset and found that LLMs can execute correct single deduction steps but fail at proof planning — they greedily follow the first applicable rule even when other branches are required. The reasoning-model era. Reasoning Models (LLM) such as OpenAI's o1, DeepSeek-R1, and QwQ explicitly trade test-time compute for accuracy by generating long internal chains-of-thought with backtracking and reflection, and report large gains on novel competition math and code problems that are unlikely to be in training data. Whether this constitutes "real" reasoning is still contested — current evidence suggests it is partially real (gains survive variant perturbation better than non-reasoning models) and partially scaled-up pattern search. The practical implication for evaluation is that any benchmark whose items are likely contaminated, or whose templates are easy to rewrite, should be paired with symbolic variants before its scores are trusted.