What Are AI Hallucinations and How Do You Reduce Them?

You ask an AI assistant for 3 sources to back up a sales proposal. It gives you 3 neat citations, complete with titles, authors and dates. Two of them do not exist.

That is an AI hallucination, and it is the single biggest reason teams get burned by generative AI. The problem is not that the tools are useless. It is that they are wrong in a way that looks exactly like being right. This guide explains why models make things up, shows real cases where it caused trouble and gives you a practical checklist to catch errors before they reach a customer, a court or your books.

What an AI hallucination is

An AI hallucination is output from a generative AI system that is false, fabricated or unsupported, yet presented as fact. Common examples include:

• Invented statistics, survey results or market sizes
• Fake quotes attributed to real people
• Citations to articles, studies or court cases that do not exist
• Wrong dates, prices, product specs or policy details
• Summaries that add points the original document never made

The US National Institute of Standards and Technology (NIST) uses the more formal term confabulation. In its Generative AI Profile (NIST AI 600-1, July 2024), NIST describes it as the production of confidently stated but erroneous or false content, known colloquially as “hallucinations” or “fabrications,” by which users may be misled. NIST lists it as one of 12 risks that are unique to or made worse by generative AI.

Why AI models make things up

Hallucinations are not a bug that a single update will remove. They come from how large language models work.

1. Models predict plausible text, not true text. An LLM writes by predicting the most likely next token based on patterns in its training data. NIST’s profile points out that this statistical prediction can produce accurate output, but it can also produce output that is factually wrong or internally inconsistent. The model has no built-in fact-checker.

2. Training and testing reward guessing. In a September 2025 paper, “Why Language Models Hallucinate,” researchers from OpenAI and Georgia Tech argued that common training and evaluation methods reward guessing over admitting uncertainty. Think of a multiple-choice exam with no penalty for wrong answers: guessing always beats leaving it blank. Models tuned to score well on such tests learn to bluff.

3. Gaps in what the model knows. A model only knows what was in its training data, up to a cutoff date. Ask about your company’s refund policy, last week’s news or a niche regulation, and it may fill the gap with something that sounds right.

4. Vague or leading prompts. “Give me 5 studies proving our product category grows every year” pressures the model to produce 5 studies, whether or not they exist.

5. Long, open-ended tasks. The longer and more detailed the output, the more chances for an error to slip in. NIST notes the risk is especially relevant for long-form answers and specialized domains.

Key takeaway: AI hallucinations happen because models are built to sound plausible, not to verify truth. Treat every specific fact, number, name and citation in AI output as unconfirmed until a person checks it against a real source.

Real cases where hallucinations cost money

These are not hypothetical risks.

Fake legal citations. In Mata v. Avianca, a federal case in the Southern District of New York, lawyers filed a brief containing court decisions that ChatGPT had invented. In June 2023 Judge P. Kevin Castel sanctioned the lawyers and their firm $5,000, finding they had submitted non-existent opinions and then stood by them after the court questioned their existence. Courts around the country have since dealt with similar filings.

A chatbot’s wrong policy answer. In February 2024, British Columbia’s Civil Resolution Tribunal ruled in Moffatt v. Air Canada that the airline was responsible for incorrect bereavement-fare information its website chatbot gave a customer. The tribunal rejected the idea that the chatbot was responsible for its own statements and ordered Air Canada to pay damages.

The lesson from both: the organization that publishes or relies on AI output owns the consequences, not the software.

Where hallucinations are most likely, and most costly

Practical ways to reduce AI hallucinations

You cannot drive hallucinations to zero, but you can cut them sharply and catch most of what slips through.

Give the model the facts

The most effective fix is to stop asking the model to remember and start asking it to read. Paste in the policy, contract, product sheet or report, and tell it to answer only from that material.

At scale, this is done with retrieval-augmented generation (RAG), which automatically searches your documents and feeds the relevant passages to the model before it answers. It is the standard design for internal help desks and customer-facing assistants.

Write prompts that allow “I don’t know”

• Tell the model: “If the answer is not in the provided material, say you do not know.”
• Ask it to quote the exact passage that supports each claim.
• Avoid prompts that assume a conclusion, such as “prove that…”
• Break large tasks into smaller steps you can check one at a time.

Ask for citations, then check them

Citations are only useful if someone clicks them. Require links or document references, and verify that each one exists and actually says what the model claims. Fabricated citations often look perfectly formatted.

Use tools instead of memory

Assistants connected to web search, calculators, databases or your CRM can look up facts instead of guessing them. This is a core reason teams are adopting AI agents with tool access, though tools add their own risks and still need oversight.

Narrow the job

A model asked to “write our annual report” will invent more than one asked to “turn these 6 bullet points into a 150-word paragraph.” Smaller, well-defined tasks with clear inputs produce more reliable output.

Keep people accountable

Assign a named reviewer for any AI-assisted content that goes to customers, regulators, courts, investors or the public. NIST also warns about automation bias, the tendency to over-trust automated systems, so train reviewers to check rather than skim.

A pre-publish checklist for AI output

Use this before anything AI-assisted leaves your team:

1. Numbers: Does every statistic, price and date trace back to a real, current source?
2. Names and quotes: Are people, companies and quotes real and accurately attributed?
3. Citations: Does every link or reference exist, and does it say what the text claims?
4. Source match: For summaries, does the output include anything not in the original?
5. Policy match: Does any customer-facing answer match your current approved policy?
6. Expert review: Has a qualified person reviewed any legal, medical, financial or tax content?
7. Disclosure: Have you followed your own rules, and any applicable laws, on disclosing AI use?

Where to start

If your team already uses generative AI, start this week:

1. List where AI output is used across your business, from marketing drafts to customer chat.
2. Rate each use as low, medium or high risk using the table above.
3. Add grounding to the high-risk uses: approved documents, RAG or tool access.
4. Write a one-page AI policy covering approved tools, data rules and review requirements. Our guide to generative AI for small business includes practical starting points.
5. Review incidents monthly. Track errors you caught and adjust prompts, sources or controls.

The bottom line

AI hallucinations are a built-in side effect of how generative models write: they predict what sounds right, and sometimes what sounds right is false. Courts and tribunals have already made clear that the business relying on the output carries the responsibility. Ground your AI in real sources, design prompts that allow uncertainty, verify every specific claim and keep a person accountable. Do that, and AI becomes a fast, useful drafter rather than a liability. For more AI terms, see our glossary.

Sources

• NIST AI 600-1: Artificial Intelligence Risk Management Framework, Generative AI Profile — definition of confabulation, its causes and automation bias
• arXiv: Why Language Models Hallucinate (Kalai, Nachum, Vempala and Zhang, 2025) — how training and evaluation reward guessing over uncertainty
• CourtListener: Mata v. Avianca, Opinion and Order on Sanctions (S.D.N.Y., June 22, 2023) — sanctions for fabricated AI-generated case citations
• Civil Resolution Tribunal of British Columbia: Moffatt v. Air Canada, 2024 BCCRT 149 — company liability for its chatbot’s incorrect information