Model hallucinations: why they happen and how to reduce them

Model hallucinations are plausible but false statements generated by a language model, delivered with the same confidence as a true answer. OpenAI defines them in almost those words and says they remain "a fundamental challenge for all large language models", including its own. Its 2025 research paper argues they persist because the way models are trained and graded rewards guessing over admitting uncertainty. This guide explains why hallucinations happen, with what OpenAI and the paper's authors say, and then how to reduce them in practice with the techniques Anthropic documents, all checked on October 6, 2026.

What is a model hallucination?

A hallucination is a confident answer that is not true. OpenAI's example shows how ordinary it looks: when researchers asked a widely used chatbot for the title of the PhD dissertation of one of the paper's authors, it confidently produced three different answers, none of them correct, and when asked for his birthday it gave three different dates, all wrong. The error is not a rare glitch in odd questions; it appears on seemingly straightforward ones, which is why a fluent answer is not evidence that it is right. OpenAI says its newer GPT-5 model has significantly fewer hallucinations, especially when reasoning, but that they still occur.

Why do language models hallucinate?

OpenAI and the paper's authors give two causes that work together.

First, how models learn. Language models first learn through pretraining, which is predicting the next word in huge amounts of text. There are no true or false labels attached to each statement: the model sees only examples of fluent language. Patterns such as spelling and parentheses are consistent, so errors there disappear with scale. But arbitrary, low-frequency facts, like a person's birthday, cannot be predicted from patterns alone, and those lead to hallucinations. The paper puts it as a classification problem: if incorrect statements cannot be told apart from facts, hallucinations will arise through natural statistical pressures.

Second, how models are graded. OpenAI's explanation is a multiple-choice test. If you do not know the answer, a wild guess might get lucky, while leaving it blank guarantees a zero. When models are scored only on accuracy, the percentage of questions answered exactly right, they are encouraged to guess rather than say "I don't know". If a model guesses a birthday, it has a 1-in-365 chance of being right; saying it does not know scores nothing. Over thousands of questions, the guessing model looks better on the scoreboard. OpenAI argues that accuracy-only scoreboards dominate leaderboards and model cards and push developers to build models that guess.

OpenAI gives a measured example from the GPT-5 system card, on the SimpleQA test:

Table: two OpenAI models on SimpleQA (from OpenAI's post of September 5, 2025)

The older o4-mini scores slightly higher on accuracy, but it gets almost three times as many answers wrong, because it almost never says it does not know. These are the figures of two models from 2025, not of today's, and they come from one test; the point is the pattern, not the ranking.

Are hallucinations inevitable?

OpenAI says no, with two caveats. They are not inevitable because models can abstain when uncertain. But accuracy will never reach 100%: some real-world questions are inherently unanswerable, whatever the model size, search or reasoning ability. It adds that a small model can find it easier to know its limits than a larger one that knows a little about a subject, because being calibrated takes less computation than being accurate. Anthropic is equally cautious: its documentation says the techniques below "significantly reduce hallucinations" but "don't eliminate them entirely". The fix OpenAI proposes is on the grading side: penalize confident errors more than uncertainty and give partial credit for appropriate expressions of uncertainty.

How can you reduce hallucinations?

Anthropic documents a short list of techniques you can apply in any prompt or application:

1. Let the model say "I don't know". Give explicit permission to admit uncertainty. Anthropic says this simple step can drastically reduce false information.
2. Ground long-document tasks in quotes. For documents over about 20,000 tokens, ask for word-for-word quotes first and only then the task. This anchors the answer in the actual text.
3. Ask for citations and verify them. Have the model cite a quote or source for each claim, or find a supporting quote after it answers; if it cannot find one, it must retract the claim.
4. Restrict it to your documents. Tell the model to use only the information you provided and not its general knowledge.
5. Compare several runs. Run the same prompt several times: inconsistencies across outputs can indicate hallucinations.
6. Iterate. Use the output as the input to a follow-up prompt asking the model to verify or expand its statements, which can catch inconsistencies; and review the reasoning when an answer looks wrong.

Giving the model the source text is also the idea behind retrieval: see what RAG is, and for how to phrase these instructions, prompt engineering basics. Long prompts have their own limits: accuracy can fall as the window fills, as explained in what a context window is.

When do hallucinations matter most?

Anthropic's advice is to always validate critical information, especially for high-stakes decisions. Our judgment on where that applies: legal, medical and financial answers, anything with numbers or dates, and names, quotations and references, which are the arbitrary facts that OpenAI's analysis says are hardest. If you cannot check an answer against a source, treat it as a lead to investigate, not as a fact. In a company, that is one more reason to keep a person in the loop for any action that is hard to undo, as discussed in agents for business.

A short checklist

• Ask the model to say when it is not sure, and to quote its source.
• Paste in the document you want it to use and tell it to rely only on that.
• For anything that matters, check the answer against a primary source.
• Run important prompts more than once and compare the answers.
• Do not take fluency as proof: a confident tone is not evidence.