A large language model hallucinates when it produces text that sounds confident and fluent but is factually wrong or unsupported by its inputs. The root cause is not a bug in any particular model — it is a direct consequence of the training objective. Autoregressive language models are optimized to predict the next token from a distribution over all possible continuations, not to track whether those tokens correspond to anything true. Fluency and truth are correlated in the training data, but they are not the same signal, and every modern LLM exploits the gap.
What "Hallucination" Actually Means
The term entered NLP research long before ChatGPT. In the widely cited Survey of Hallucination in Natural Language Generation (Ji et al., ACM Computing Surveys, 2023), the authors define hallucination as generated content that is "nonsensical or unfaithful to the provided source content" and distinguish two subtypes that remain the standard vocabulary in 2026:
- Intrinsic hallucination — the output contradicts the input. A summarizer given an article about a 40-megawatt reactor writes "400 megawatts."
- Extrinsic hallucination — the output introduces information that cannot be verified from the input at all. A question-answering system invents a citation, a court case, or a co-author who does not exist.
A second useful distinction, borrowed from cognitive science, separates fabrication from confabulation. Fabrication is the generation of information with no basis in training data or input. Confabulation is the retrieval of something the model has seen but stitches together incorrectly — a real author attributed to the wrong paper, a real drug described with the wrong dosage. Both look identical to a user, but they require different mitigations.
The recent HALoGEN benchmark (Ravichander et al., 2025) formalizes this into three error types: Type A errors from incorrect recollection of training data, Type B errors from incorrect knowledge already in training data, and Type C errors from pure fabrication. Evaluating roughly 150,000 generations from 14 language models, HALoGEN found that even the best-performing models produced hallucinated atomic facts in up to 86% of outputs in some domains.
Why Next-Token Prediction Creates Hallucination
A decoder-only transformer is trained on one objective: given a sequence of tokens, assign probability to the next token. During training, the only feedback is how closely the predicted distribution matches the actual next token in the corpus. There is no separate signal for "is this true," no penalty for confident wrongness, and no reserved token that means "I do not know this."
Andrej Karpathy has argued that hallucination is, in a strict sense, all that LLMs do. The model is a dream machine whose dreams happen to be useful when the prompt is well-matched to the training distribution. Hallucination is not a failure of the mechanism; it is the mechanism viewed from a different angle. What we colloquially call a hallucination is simply a dream that crossed into territory the user cares about being true.
Several properties of next-token training make hallucinations predictable:
- No explicit abstention token. Saying "I don't know" during pretraining is almost always wrong relative to the ground-truth next token, because the corpus rarely contains that phrase where a factual answer is expected. The model is trained out of abstention.
- Memorization gaps. Facts seen once or twice during training leave weak traces in parameter space. When queried, the model interpolates between similar-sounding training examples rather than retrieving the exact fact.
- Pressure to continue. Autoregressive decoding commits to a token at each step and conditions all subsequent tokens on that choice. A single plausible but wrong token pulls the entire continuation toward a confabulated trajectory.
- Calibration is not accuracy. A model can be well-calibrated (its confidence matches its accuracy on average) while still producing highly confident errors on individual prompts.
The 2025 paper "Why Language Models Hallucinate" by Kalai, Nachum, Vempala, and Zhang (OpenAI and Georgia Tech) makes this rigorous. They show that standard training and evaluation reward guessing over admitting uncertainty: when "I don't know" is scored the same as a wrong answer, the expected-reward-maximizing strategy is to guess. They prove a generation-classification inequality stating that the generation error rate is at least twice the corresponding classification error rate minus a calibration term — meaning some hallucination is a direct mathematical consequence of the current training recipe, not an artifact of model size or data quality.
Why Scaling Alone Hasn't Solved It
The most counterintuitive finding in the factuality literature is that bigger models sometimes hallucinate more, not less. TruthfulQA (Lin, Hilton, and Evans, 2021) was the first benchmark to document this clearly. Across 817 questions spanning 38 categories of common human misconceptions, the best GPT-3 variant answered truthfully on only 58% of questions compared to 94% for humans — and within the GPT-3 family, larger models were generally less truthful than smaller ones. The authors interpreted this as "inverse scaling": bigger models imitate the training corpus more faithfully, including its false beliefs.
Subsequent benchmarks confirmed that scale is an incomplete solution.
- HaluEval (Li et al., 2023) generated 30,000 task-specific hallucinated samples and found that ChatGPT fabricated unverifiable content in roughly 19.5% of user queries in its test set.
- FELM (Chen et al., NeurIPS 2023) annotated 4,425 text segments from LLM outputs and reported a response-level error rate of 33.3% overall, rising to 46.2% in the world-knowledge domain. Even GPT-4 achieved only 48.3 F1 at detecting its own factual errors.
- OpenAI's SimpleQA (Wei et al., 2024), a benchmark of 4,326 short-form factual questions built so that each one has a single indisputable answer, showed GPT-4o and Claude 3.5 Sonnet scoring below 50% and o1-preview peaking at 42.7% correct. The paper explicitly notes that models "consistently overstate their confidence."
- Google DeepMind's FACTS Grounding (2024), which tests whether long-form responses stay faithful to a supplied source document, reports top scores in the 83-91% range for frontier models — meaning roughly one response in ten is still ungrounded even with the source explicitly provided.
Stanford's HELM (Liang et al., 2022) made a broader point by evaluating 30 models across 42 scenarios on seven axes including accuracy and calibration. The HELM results show that accuracy and calibration improve unevenly with scale, and that no single model dominates across scenarios. Factuality is not one problem; it is a family of problems that each respond differently to more parameters.
What Actually Reduces Hallucinations in 2026
No single technique eliminates hallucination. Current best practice combines several that attack different parts of the problem.
Retrieval-augmented generation (RAG). Introduced by Lewis et al. at NeurIPS 2020, RAG pairs a parametric language model with a non-parametric memory — typically a dense vector index over a trusted corpus. At inference time, the user query is embedded, the index returns the top-k most relevant passages, and those passages are prepended to the prompt as context. The LLM then generates grounded in retrieved text rather than from its parameters alone. RAG became the industry default because it addresses the most common failure mode: questions about facts the model never memorized well in the first place. It does not fix hallucination in reasoning, and it introduces new failure modes when retrieved passages are themselves wrong or irrelevant.
Constitutional AI and RLAIF. Anthropic's Constitutional AI paper (Bai et al., 2022) trains a model to critique and revise its own outputs against a written set of principles, then uses those revisions as a preference signal for reinforcement learning from AI feedback. Constitutional methods can target honesty and calibration directly by including principles that penalize overconfident assertions. They do not eliminate hallucination, but they shift the model's default toward hedging where appropriate.
Process reward models. In Let's Verify Step by Step (Lightman et al., 2023), OpenAI showed that training a reward model to score each intermediate reasoning step — rather than only the final answer — substantially outperforms outcome-only supervision on mathematical problems. Process supervision reduces a particular class of hallucinations: the confident but wrong intermediate step that poisons the rest of a chain of thought. The released PRM800K dataset contains 800,000 step-level human feedback labels and has become a reference for process-reward research.
Calibration fine-tuning and behavioral abstention. The OpenAI Why Language Models Hallucinate paper proposes a concrete fix: penalize confidently wrong answers more heavily than uncertain ones during post-training, and give partial credit for well-calibrated expressions of uncertainty. Early results suggest that behavioral calibration — training the model to abstain or hedge when internal confidence is low — reduces confident hallucinations without collapsing model usefulness.
Hallucination evaluators as guardrails. Production systems increasingly route LLM outputs through a second model trained specifically to detect hallucination. Vectara's HHEM leaderboard and Google's FACTS Grounding leaderboard both use an evaluator-LLM approach and have become reference metrics for RAG faithfulness in 2025 and 2026.
The Remaining Limits
Even with RAG, process reward models, and calibrated post-training, several failure modes resist current techniques.
- Knowledge cutoffs. A parametric model cannot know about events after its training data. RAG mitigates this only if the retrieval corpus is kept fresh, which most production stacks do not guarantee.
- Multi-hop reasoning errors. Hallucinations in chain-of-thought reasoning compound: one fabricated intermediate fact corrupts every downstream step. Process reward models help but do not eliminate the problem, and the boundary between a reasoning error and a factual hallucination is often unclear. The broader question of when a model's reasoning can be trusted is one of the hardest open problems in AI safety — see our Research Roadmap on reasoning reliability for the current state of research.
- Long-context drift. As context windows grow into the millions of tokens, models show a distinct "lost in the middle" effect: facts placed halfway through a long prompt are attended to less reliably than those at the start or end.
- Retrieval failures. RAG is only as good as its retriever. When the embedding model returns irrelevant passages, the LLM often proceeds anyway and produces a confident answer grounded in the wrong source.
- Hallucinations that align with the retrieved document. If the corpus itself contains errors — outdated medical guidelines, a retracted paper, a scraped Wikipedia revision — the RAG system will faithfully repeat them.
These limits motivate the continued push toward mechanistic interpretability, which aims to understand what LLMs actually compute internally so that fabrication can be detected before it reaches the output tokens.
Frequently Asked Questions
Can a language model tell when it is hallucinating?
Current evidence suggests partially. Internal activations correlate with the model's own uncertainty, and probes trained on hidden states can detect some hallucinations better than chance. But as the FELM results show, even GPT-4 asked to judge its own outputs achieves only about 48 F1 at detecting factual errors. Self-evaluation is a useful signal, not a solution.
Does retrieval-augmented generation eliminate hallucination?
No. RAG reduces hallucinations that arise from missing parametric knowledge, but it introduces new failure modes: irrelevant retrievals, stale corpora, and hallucinations that sound faithful to a retrieved passage but actually misquote it. The FACTS Grounding benchmark shows that frontier models remain ungrounded on roughly 9-17% of responses even when the source document is supplied.
Why do bigger models sometimes hallucinate more?
The TruthfulQA result showed that larger models within the same family can be less truthful, because they imitate human-written falsehoods more faithfully. Subsequent work has shown that scaling helps on some factuality benchmarks and hurts on others. Scale alone is an incomplete intervention.
What is the difference between hallucination and a reasoning error?
Hallucination usually refers to a factual claim that is wrong or unsupported. A reasoning error is a logical mistake — a miscomputation, an invalid deduction, a misapplication of a rule. In practice the categories blur: a fabricated intermediate fact in a chain of thought is both. The HALoGEN taxonomy (Type A, B, and C) is one attempt to sort these into distinct categories for measurement.
Will hallucinations be solved by 2030?
Probably not in the sense of being eliminated. The OpenAI "Why Language Models Hallucinate" paper gives a mathematical reason to expect some residual hallucination rate as long as models are trained with next-token prediction and evaluated with grading schemes that reward guessing. Reducing the rate substantially and making remaining hallucinations detectable are realistic goals. Eliminating them entirely is not, on current methods.
Key Takeaways
- Hallucination is a direct consequence of the next-token-prediction training objective, not a bug specific to any model family.
- Scale alone does not solve the problem: TruthfulQA, HaluEval, FELM, SimpleQA, and FACTS Grounding all show frontier models still hallucinating at meaningful rates as of 2026.
- Retrieval-augmented generation is the most effective single mitigation but introduces its own failure modes and does not address reasoning errors.
- Process reward models, constitutional training, and calibration fine-tuning each reduce specific subtypes of hallucination; combining them is current best practice.
- Mechanistic interpretability and behavioral abstention are the most promising directions for making residual hallucinations detectable before they reach the user.
The Path Forward
Hallucination is not going away in 2026, but the research landscape has matured from panic to measurement. We have benchmarks that distinguish intrinsic from extrinsic errors, taxonomies that separate fabrication from confabulation, and mitigation techniques that each address a specific failure mode rather than pretending to solve the whole problem at once. The honest assessment is that any system deployed in a high-stakes setting needs retrieval, evaluation, and a human in the loop — and will continue to need all three for the foreseeable future.
At DeepScience, we track the latest research on LLM factuality and grounding. Our Research Roadmap covers hallucination and grounding alongside mechanistic interpretability as key open problems in AI safety.