Definition

Hallucination is when an AI model, especially a large language model, produces content that reads as fluent and credible but is actually false, contradicts the material it was given, or is invented outright. Typical cases include citing papers that don’t exist, giving wrong dates or figures, describing functions a library doesn’t have, and blending two people’s biographies.

What makes hallucination dangerous is that a model sounds just as confident when it’s wrong as when it’s right, so users can’t tell truth from fiction by tone alone.

Research commonly splits hallucination into two kinds:

  • Factuality hallucination: content that contradicts real-world facts.
  • Faithfulness hallucination: content inconsistent with the user’s material or instructions, such as adding claims to a summary that weren’t in the source.

How It Works

Hallucination isn’t a glitch. It follows from how language models work:

The goal is plausible, not correct. A language model is trained to predict the most likely next token, so it learns the statistical patterns of language. Text that is fluent and fits familiar patterns can be generated even when it’s wrong.

Knowledge is stored in compressed form. A model doesn’t keep facts one by one like a database; it compresses vast amounts of information into its parameters. For rare knowledge that appears seldom in training data, its memory is often fuzzy, leading it to mix things up or fill in gaps.

Training data has errors and goes stale. Mistakes, contradictions and outdated information in the data get learned too.

A bias toward answering. If training and alignment consistently reward a complete answer over “I don’t know,” the model learns to answer even when unsure.

Sampling randomness. Generation involves some randomness. Higher temperature means more varied output and a greater chance of drifting from the facts.

Typical Applications

Here, “applications” means practical ways to spot and reduce hallucination:

  • Retrieval-augmented generation (RAG): have the model answer from retrieved material and cite sources.
  • Citation checks: verify every paper, link and number the model provides.
  • Prompt constraints: explicitly say “if the material doesn’t cover it, say you don’t know.”
  • Tool use: send calculations to a calculator or code, and current events to search, instead of relying on memory.
  • Lower temperature: use lower sampling temperature for factual tasks.
  • Compare multiple samples: ask the same question several times; inconsistent answers flag what to check.
  • Human review: in high-stakes fields like medicine, law and finance, model output needs expert review.

No current method eliminates hallucination entirely. The realistic goal is reducing it to an acceptable level.

  • Large language model: where hallucination is most discussed.
  • RAG: a common way to reduce factuality hallucination.
  • RLHF: alignment training can shape a model’s willingness to admit uncertainty.
  • Context window: with very long material, models can miss or confuse details, causing faithfulness problems.
  • Prompt engineering: constraints and examples help cut down on fabrication.

Survey: Survey of Hallucination in Natural Language Generation