Ask a support chatbot about a discount code that doesn’t exist, and a well-behaved one says so. A hallucinating one invents the code, the terms, and an expiry date — fluently, confidently, and in your brand’s voice. Your customer screenshots it. Now it’s a refund dispute.
Hallucination is the failure mode that makes LLMs different from every system QA has dealt with before: the software doesn’t crash, throw an error, or log a warning. It succeeds — at producing something false. Hallucination testing is how you find that failure before your users do.
What exactly is a hallucination?
A hallucination is output that is fluent and plausible but factually wrong or unsupported by any source the model was given. It’s not a bug in the code — it’s intrinsic to how language models work. An LLM predicts likely-sounding text; when it doesn’t know an answer, “likely-sounding” and “true” can quietly part ways. That means you cannot fix hallucination once and move on. You can only measure it, reduce it, and watch it.
The three flavors QA teams should distinguish
- Open-world hallucination — the model invents facts about the world: fake statistics, fake citations, events that never happened.
- Grounding failure — in RAG systems, the model ignores or contradicts the documents it retrieved, answering from its general training instead of your data. It may be “right about the world but wrong about your business.”
- Instruction hallucination — the model claims it did something it didn’t: “I’ve escalated your ticket,” says the bot with no ticket system access.
They have different causes and different fixes — a test suite that lumps them together will tell you something is wrong but not what to do about it.
How hallucination testing works in practice
Trap questions
Build test cases whose correct answer is “that doesn’t exist” or “I don’t know”: products you never sold, policies you never wrote, colleagues who never existed. A model that answers anyway is hallucinating on demand — and now you can count how often.
Groundedness checks
For RAG systems, every claim in the answer is checked against the retrieved documents: is this sentence actually supported by the source? This can be automated with an evaluator model scoring claim-by-claim, and it catches the subtle case where the answer is 90% grounded and 10% invented — the hardest kind for a human reviewer to spot.
Factuality scoring against a golden dataset
For questions with known correct answers, responses are scored against expected properties — does it mention the real 30-day window, the real price, the real escalation path? Run on every release, this turns “hallucination” from an anecdote into a metric: a rate you can track, gate releases on, and actually improve.
Consistency probing
Ask the same question five ways — rephrased, with typos, in a different order. A grounded model gives consistent answers; a hallucinating one drifts. Inconsistency is one of the cheapest hallucination signals you can automate.
What “good” looks like
No serious team ships at zero hallucinations — the honest goal is a measured rate, on your real question distribution, that fits your risk. An internal dev-tools bot can tolerate more than a bot quoting medication doses. What matters is that the number exists, someone owns it, and it’s re-measured on every prompt change, model upgrade, and knowledge-base update — because any of those can move it silently.
Where to start
- Write 20 trap questions for your product today — things that don’t exist but sound like they could
- Pull your top 50 real user questions and define what a correct answer must contain
- If you use RAG, spot-check ten answers claim-by-claim against the retrieved sources
- Automate whatever you did manually, and run it on every release — see our full pre-launch chatbot checklist
Or let us build it for you
Hallucination and factuality testing is a core part of our AI & LLM testing service — golden datasets, groundedness evals, and a hallucination rate you can actually see on a dashboard. Book a free strategy session and we’ll look at your AI product together. It’s free, and the whole conversation is about you.