Does AI Really 'Think'? — A Question Researchers Are Still Fighting Over

The Answer Splits Depending on Definition and Test

When given a difficult problem, AI sometimes does not spit out an answer right away, but instead lays out its steps: "First I calculate this, then I consider that…." Chain-of-thought is when AI writes out intermediate working steps one sentence at a time before giving an answer. In English, it is Chain-of-Thought, abbreviated as CoT. Watching those sentences pile up on the screen makes it seem like the machine is truly "thinking."

"AI Models — A Calculation That Learns From Examples Instead of Rules" left us with the question of whether generalization—getting a good score on unseen data—is really understanding. The primer called the actual use of a fully trained model inference. Inference in this article is different: it means stepping through given facts to draw a conclusion, and even its precise meaning is a matter of dispute. This article follows that question, narrowing it down to inference in this sense.

The academic community has not yet reached a consensus on whether AI reasons. The answer splits depending on how reasoning is defined and what kind of problems are used to test it, and researchers still refute one another with papers.

The Skeptics: "It May Be Mimicking Learned Solutions"

The evidence from the skeptics comes from experiments that slightly twist problems. When tested with elementary-level math problems where only the numbers were changed, all the latest 2024 models saw their scores drop, and inserting just one sentence that was unrelated to the answer yet looked relevant caused performance to plunge by up to 65%. As the number of condition clauses in a question increased, performance degraded significantly. The authors hypothesized that the root cause is that "current models do not perform genuine logical reasoning and instead replicate the reasoning steps of their training data" (arXiv:2410.05229).

A similar debate surrounded analogy. To the claim that GPT-3 spontaneously acquired the ability to solve analogy problems without examples, a rebuttal study presented a counterexample using string analogy tasks that required finding and transferring rules between lines of text. Even on the simplest version where the original task was altered just slightly, GPT-3 failed while humans maintained high scores. The authors emphasized that "methodology to rule out data memorization is needed" (arXiv:2308.16118).

Suspicion of benchmark contamination adds to the mix. As described in "How AI Gets Things Wrong — The Test-Taker Who Stumbles Outside the Exam Hall", benchmark contamination is when exam questions or very similar texts get mixed into the training data beforehand. This makes it hard to distinguish whether the model solved the problem or simply produced a memorized answer. One collaborative project gathered hundreds of contamination cases into a public database, exposing the reliability issue of evaluation results (arXiv:2407.21530). Even many benchmarks measuring causal inference—which weighs cause and effect—can likely be solved just by pulling in domain knowledge, raising doubts about whether they truly measure reasoning (arXiv:2407.08029).

Weaknesses are exposed when conditions are messy. Models fine-tuned with reinforcement learning—a method of correcting answers by giving scores to attempts—saw their reasoning scores rise under ideal conditions, but their performance dropped sharply across all three "non-ideal" situations: summary inference, noise suppression, and context filtering. While the authors proposed situation-specific patches, they found that current methods generally leave this deficit intact, concluding that "the reasoning capability of large models is often exaggerated" (arXiv:2508.04848). Even the task of word "letter composition," which humans execute perfectly, could not be handled reliably by most of the models tested in 2024 (arXiv:2405.11357).

An "Illusion of Thinking," Or Just a Lack of Tools?

Reasoning models are models made to write out a long chain of thought before giving an answer. Right at the center of the debate surrounding these models is a 2025 study called "The Illusion of Thinking." According to a commentary covering this research, the original study found a "reasoning cliff"—where reasoning model performance collapses like a cliff once problem complexity crosses a certain threshold—and interpreted this as an inherent limit that chain-of-thought reasoning cannot overcome even by scaling up.

While acknowledging that the original study's methods are rigorous, the commentary reads this collapse differently. It argues that the breakdown is the result of overlapping constraints in evaluation environments that only allow writing: namely, the ban on using tools, the inability to retrieve long context, the absence of a core cognitive baseline, inadequate statistical reporting, and output length limits. The commentary reports that one model which called a puzzle "impossible" when restricted to writing alone solved that puzzle when handed tools, even tackling difficulties beyond the reasoning cliff. The action gap is the name this commentary gives. The commentary views this performance collapse as a gap caused by failing to execute when the model's environment blocks the tools it needs to carry out its solution (arXiv:2506.18957). Reading the same fact as an "inherent limit" on one side and as "fault of the environment" on the other is typical of the debates in this field.

Capability Side: "Actually Solving Harder Problems"

Evidence is also accumulating on the capability side. Making models think longer before answering raises scores on tricky tasks. A survey on long chains of thought summarizes that deep reasoning, broad search, and looking back to check help solve complex problems, while noting that debates persist around "excessive thinking" and ways that spend more computation on answering (arXiv:2503.09567).

In a study that systematically evaluates 12 models for the first time across 17 Chinese and English legal tasks, DeepSeek-R1 clearly led on Chinese tasks while o1 delivered comparable scores on English tasks, and a law-specific model built by the authors also achieved competitive scores across multiple tasks. However, error analysis repeatedly showed outdated legal knowledge, limits in legal interpretation, and hallucinations that fabricate facts (arXiv:2503.16040).

Sleep-time compute is a method that pre-computes given context before a question arrives. This method achieved the same accuracy using 5 times fewer computations at the time of answering, and increasing pre-computation raised accuracy by up to 13% and 18% respectively across two tasks. However, the effect was strongly tied to how well the user's future questions could be predicted (arXiv:2504.13171).

There is also an evaluation comparing models with humans using an educational higher-order thinking test. The reasoning model o1-preview outperformed humans in most categories, and the authors report that it yielded results 150% better than humans in systems thinking, computational thinking, data literacy, creative thinking, scientific reasoning, and abstract reasoning. The abstract does not disclose the human group compared or the baseline for this ratio. On the other hand, it lagged by about 25% in logical reasoning, critical thinking, and quantitative reasoning, while both humans and models scored a perfect mark in analogical reasoning. However, the same abstract includes abstract reasoning in the list of advantages while also noting that psychology majors outperformed the model in that category, making the report's findings internally inconsistent for this category (arXiv:2410.21287). Reasoning capability is uneven across tasks rather than being a simple yes-or-no trait. In a pilot study examining planning capability, o1-preview outperformed GPT-4 in structured environments through constraint compliance and state management, but it frequently produced solutions mixed with unnecessary actions and was weak at generalization on spatially complex tasks (arXiv:2409.19924). In a symbolic reasoning benchmark, reasoning models handled longer relationships and broader attributes better than regular language models, but they still got severely blocked on problems mixed with uncertainty (arXiv:2510.17496).

The Subtle Trap: On-Screen Working and Actual Computation Can Diverge

Faithfulness is the degree to which the step-by-step working a model writes out actually reflects the process that led it to that answer. A 2023 study noted that the on-screen chain of thought does not necessarily show the path that led to the answer. As an alternative, it proposed an approach where the language model merely translates the question into symbolic operations and leaves the computation to an external solver that follows strict rules. This approach outperformed standard chain-of-thought on 9 out of 10 benchmarks across four domains, showing that faithfulness and accuracy can go hand in hand (arXiv:2301.13379). This kind of "unfaithful reasoning" was also classified as a type of deceptive behavior in an essay-style review (arXiv:2403.09676).

There are also studies showing that the reasoning behavior itself corresponds to structures deep inside the model. A study that looked inside a set of "thinking" models showed that reasoning behaviors like expressing uncertainty, generating examples, and backtracking correspond to specific directions in the space formed by the numbers inside the model. A steering vector is a value extracted from that direction and added or subtracted to amplify or reduce that behavior (arXiv:2506.18167). This means that at least in those models, there is a detachable, adjustable structure behind the working-out behavior. What this study measured was the structure that controls the working-out behavior, while whether the written-out working matches the actual computation remains a separate question.

So — What to Check First Is the Word "Think"

The biggest trap may be the word itself. When a machine strings together a solution, we compare it to a human and say it "thinks," but what benchmarks measure is whether it gets the problem right, and whether it thinks like a human is not included in the score. "Emergent abilities" originally referred to abilities that do not exist in small models but appear in large models, making them unpredictable by extending the performance curves of small models (arXiv:2206.07682). The earlier analogy rebuttal stated that claiming human-like reasoning requires a way to rule out data memorization (arXiv:2308.16118). A long-horizon reasoning benchmark released in 2026 collected problems where solutions span tens of thousands to hundreds of thousands of tokens. Tokens are the pieces into which language models chop up text. At the time of release, even the best models had an accuracy of under 10%, breaking down when weaving together steps that they could each solve individually (arXiv:2604.14140).

"What looks like reasoning" and "reasoning" can be different, and research is currently debating where the two diverge. One study suggests "replication," arguing that changing even the numbers causes it to collapse (arXiv:2410.05229), while a commentary blames the "environment," claiming that simply handing it tools allows it to cross the cliff (arXiv:2506.18957). Both are looking at the same kind of machine.

---

This article covers a rapidly changing and still debated field. Much of the cited evidence consists of preprints prior to peer review, and conclusions may change later. We have tried not to definitively declare any specific stance as the right answer.

Source Papers (arXiv)

• arXiv:2410.05229 — GSM-Symbolic: Limits of Mathematical Reasoning
• arXiv:2308.16118 — A Rebuttal to Claims of 'Emergent Analogical Reasoning'
• arXiv:2407.21530 — 2024 CONDA Data Contamination Report
• arXiv:2407.08029 — A Critical Review of Causal Inference Benchmarks
• arXiv:2508.04848 — Reasoning Capabilities Under Non-Ideal Conditions
• arXiv:2405.11357 — Lack of Character Composition Understanding in LLMs
• arXiv:2506.18957 — Commentary on "The Illusion of Thought": Reasoning Cliffs vs. Action Gaps
• arXiv:2503.09567 — Survey on Long Chain-of-Thought (Long CoT)
• arXiv:2503.16040 — Inference-Time Scaling in Legal Reasoning
• arXiv:2504.13171 — Sleep-Time Compute
• arXiv:2410.21287 — Evaluating Higher-Order Thought in o1-preview
• arXiv:2409.19924 — Planning Capabilities of o1 Models
• arXiv:2510.17496 — I-RAVEN-X: Generalization and Robustness in Analogy and Mathematical Reasoning
• arXiv:2301.13379 — Faithful Chain-of-Thought (Faithful CoT)
• arXiv:2403.09676 — Deceptive Capabilities of LLMs (Unfaithful Reasoning)
• arXiv:2506.18167 — 'Thinking' Models Through the Lens of Steering Vectors
• arXiv:2206.07682 — Emergent Abilities of Large Language Models
• arXiv:2604.14140 — LongCoT: A Long-Horizon Reasoning Benchmark