Hallucination

A hallucination sounds plausible and is wrong: the model cites a study that does not exist under the title it gives, a wrong detail or a conclusion the source text does not support. The treacherous part is the form. A hallucinated answer is just as fluent and sounds just as confident as a correct one. The text itself often gives no hint of the difference.

The cause lies in how language models work. A model calculates which continuation of a text is likely, and it does not look anything up in the process. When information is missing, it fills the gap with a phrasing that sounds likely. In September 2025, OpenAI researchers added another explanation: standard training and evaluation methods reward guessing more than admitting uncertainty.

For executives, one conclusion matters most: with today's technology, hallucinations can be reduced but not ruled out. Responsibility for a false statement stays with the company that passes it on. That is why review belongs in the workflow rather than relying solely on individual employees paying attention.

Why language models hallucinate.

A language model generates its answer one token at a time. At each step, it picks a continuation that is likely given what has been written so far. This mechanism does not check whether the resulting statement is true.

Trouble starts at the edges of what the model knows. For rare facts that barely appear in the training data, for events after the model's knowledge cutoff and for precise details such as case numbers, birth dates, page numbers or verbatim quotes, there is no reliably likely answer. The model produces one anyway, and it has the right shape.

In 2025, OpenAI researchers described another reason. Many evaluations only count correct answers. A model that guesses when unsure is sometimes right and scores better than one that says "I don't know." Their example: if you guess someone's birthday, you will occasionally hit it; if you admit you don't know, you never score. Training and testing thereby reward guessing.

The application can encourage hallucinations too. An instruction that demands an answer no matter what, a vague question or a context full of contradictory documents all raise the risk.

Two kinds: wrong about the world, wrong about the source.

Research surveys usually distinguish two kinds of hallucination. In the first, a statement is factually wrong: an invented court ruling, a wrong year, a study that does not exist. Researchers call this a failure of factuality.

In the second, the answer departs from a source the model was given. A summary contains a figure that appears differently in the report. An answer from the knowledge base draws a conclusion the source text does not support. Meeting minutes attribute a statement to the wrong person. This is called a failure of faithfulness.

The distinction matters in practice because the two are checked differently. The first is checked against an independent primary source: the ruling in the court database, the study at the publisher. The second is checked by placing the answer next to the source text.

That is why giving a model your own documents, for example through RAG, does not settle the matter. The first kind becomes rarer because the model works from the sources it is given. The second remains: the model can render a correct passage incorrectly, blend two passages together or claim more than the text says.

Where hallucinations get expensive.

How much a hallucination matters depends on where the statement goes. A wrong sentence in an internal draft that someone will rework anyway costs little. It gets expensive when a statement goes out unchecked or underpins decisions.

Two North American cases became well known. In Mata v. Avianca in 2023, lawyers filed a brief in a New York federal court citing rulings that ChatGPT had invented; the court sanctioned them. In Moffatt v. Air Canada, an airline's chatbot had given a customer wrong information about a refund. In 2024, a tribunal in British Columbia held the airline responsible and rejected its argument that the chatbot was responsible for its own statements.

Source citations and quotes are especially sensitive, as are figures in proposals and reports, legal assessments, medical and technical safety information, product information given to customers and anything that ends up in contracts or letters to authorities. In these areas, it is not enough for an answer to sound plausible.

Review should therefore scale with the stakes: the greater the consequences of a false statement, the stricter and more expert the review has to be.

What reduces the risk.

Several measures significantly reduce how often hallucinations occur. None eliminates them entirely.

The most effective is to give the model the information it needs instead of letting it answer from memory: the document, the database query, the search results. That pairs well with an explicit instruction to answer only from those sources and to say when they contain nothing relevant. It also helps to require a source location for every claim, and for quotes, the verbatim text from the source.

A narrow task definition helps as well. A model that extracts five fixed fields from an email has less room to improvise than one writing a free-form report. Calculations belong in a calculator tool, not in the text output. Settings such as a low temperature make answers more consistent, not more true.

Some applications have a second model cross-check the answer. That catches some errors, especially departures from a source the model was given. Source access and a second model check can still miss errors. Important statements need independent review, by a person who opens the source themselves.

Review belongs in the workflow.

Because hallucinations are often impossible to spot in the text, attention alone does not help much. Review that is a fixed step in the workflow is more reliable.

A simple ground rule: whoever passes on an AI result has checked it and answers for it. Review levels can be derived from that, based on the stakes. For internal drafts that will be reworked anyway, spot checks are enough. For customer-facing texts and publications, figures, names and sources are checked against the original. For legal, medical or safety-related statements, a qualified expert reviews them, and nothing goes out without their sign-off.

For source citations, one plain rule applies: a source only counts as verified once someone has opened it and read the passage. An invented source looks as if it has already been checked, because it has the form of evidence. That is exactly what makes it dangerous.

Errors that are found should be recorded: what kind, in which task, with which model. These cases become a test list against which later changes to instructions, sources or models can be measured.

What a good implementation looks like.

A good AI application does not promise to be free of hallucinations. It makes errors rare, visible and checkable. If a vendor says its system does not hallucinate, ask how that was measured.

Measurement uses a fixed list of real tasks from your own operations whose correct answers are known. The error rate is established against that list before rollout and measured again after every change. Hallucination rates from vendors' general benchmarks do not carry over to your own task; a model that scores well there can be weak on a specific domain question.

Other signs of quality: every answer cites its source, and the source opens with one click. The system is allowed to say it found nothing, and it does. Answers with consequences for customers or contracts go through human sign-off. Questions, answers and reported errors are logged so that recurring weaknesses stand out.

When talking to vendors, three questions are worth asking: On which tasks was the error rate measured? What does the system do when it finds no suitable source? And who reviews answers before they go out?

Where the term stops working

Source access and a second model check can still miss errors. Important statements need independent review. Not every AI error is a hallucination: an outdated knowledge cutoff, a wrong document in the knowledge base, a calculation error or deliberate manipulation through injected instructions (prompt injection) have different causes and need different remedies.