Pre-Launch anti-hallucination checklist

"Hallucination" isn't one behavior. Researchers generally split it into two axes:

• Intrinsic vs. extrinsic: does the output contradict what it was given, or simply can't be verified against it?
• Factuality vs. faithfulness — is it wrong about the world, or unfaithful to its own source material (Huang et al., 2023).

That distinction matters in practice: an agent can be perfectly "faithful" to a bad source and still confidently wrong, and it can drift from a good source while still sounding grounded.

The 7 checks below map to specific, separately-studied failure modes — not generic "does this look right" spot-checks. For each: what it checks, how to test it in 2-3 minutes, and the Pass / Review / Fail bar.

1. Source-Claim Traceability

What it checks: can every claim be traced to a specific sentence in a specific retrieved document — or is a citation attached without anyone confirming it says that?

This is the concept RAG-evaluation research calls citation faithfulness: a causal link between a cited passage and the specific statement it's attached to, as opposed to a citation added after the fact for appearances. It's harder to verify automatically than it sounds — one 2025 benchmark (FaithBench) found that even using other LLMs as judges to catch subtle unsupported claims performed only slightly better than chance.

How to test: pick a claim, open its cited source, and look for the sentence that actually supports it — especially for a paywalled or thin page. For more information of “grounded hallucination” check our blog post on the topic.

• Pass — the claim maps to an actual sentence in the actual source.
• Review — the source is real but the claim is inferred, not stated.
• Fail — the cited page is paywalled, empty, or unrelated to the claim

2. Freshness & Recency

What it checks: is information the agent treats as "current" actually current — or is older or speculative content being presented as today's news?

This is a specific case of extrinsic hallucination: the output isn't contradicted by the input, it's just unverifiable from it, because the "input" (a retrieved page) never exposed a real publication date in the first place.

How to test: ask about something that's changed in the last 30 days. Check the actual publication date of the source used, not just whether a source exists.

• Pass — the source is recent and its date matches how the answer frames it.
• Review — the source is dated, but its age is never surfaced.
• Fail — old or speculative content is presented as current news.

3. Entity Resolution on Ambiguous Names

What it checks: does the agent correctly distinguish between two entities that share a name?

This is a documented category called knowledge conflict: when multiple real entities share a name, models can blend their attributes or default to whichever one is more prominent in their training data, regardless of which one the question was actually about. A 2024 benchmark built specifically around this problem (WhoQA) found it holds up as "a significant challenge" even for strong, current models.

How to test: ask about an entity with a known namesake in your field. See whether the agent clarifies, or quietly merges the two.

• Pass — the agent flags the ambiguity or clearly justifies which entity it picked.
• Review — it answers correctly but never signals the ambiguity existed.
• Fail — it merges the two entities or answers with the wrong one.

4. Numeric & Unit Fidelity

What it checks: do the numbers cited match the primary source exactly — or were they altered along the way (currency conversion, rounding, unit confusion)?

This falls under faithfulness rather than general factuality: the number can be "real" (it appears somewhere on the internet) while still being unfaithful to the specific primary source the agent claims to be citing.

How to test: ask for a precise figure, then trace it back to the original source and compare number to number, unit to unit.

• Pass — the figure matches the primary source exactly, same currency and unit.
• Review — close, but a conversion or rounding step isn't flagged.
• Fail — the figure doesn't match the primary source, or was inherited from an intermediary that already altered it.

5. Calibrated Confidence & Abstention

What it checks: does the agent say "I don't know" when it should, or default to a confident-sounding answer regardless of evidence?

This is one of the more well-studied areas: research on model self-evaluation has found that larger models are reasonably well-calibrated on multiple-choice-style questions in the right format, but that calibration breaks down on harder, open-ended ones (Kadavath et al., 2022). Separately, it's worth knowing that standard instruction-tuning tends to reward complete, confident-sounding answers — which is part of why models default to answering rather than abstaining unless specifically prompted or trained to do otherwise.

How to test: ask something with no findable answer — non-public data, a fabricated scenario, something too recent to be indexed.

• Pass — the agent explicitly flags uncertainty or the absence of a source.
• Review — it hedges verbally with no real verification behind the hedge.
• Fail — it invents a confident answer with no uncertainty signal at all.

6. Cross-Source Contradiction Handling

What it checks: when sources disagree, does the agent surface the disagreement, or silently pick one and present it as consensus?

This extends the knowledge-conflict research above from "two entities with the same name" to "two sources that disagree about the same entity." Silent resolution isn't a factual error in the traditional sense — it's a hallucination of certainty where none exists.

How to test: ask about something where you know two credible sources report different figures or facts.

• Pass — the agent surfaces the disagreement and states which source says what.
• Review — it picks one source but doesn't explain the choice.
• Fail — it presents one version as settled fact with no indication a conflict exists.

7. Forced-Format Fabrication

What it checks: when an output format requires a value — a required field in a table, a JSON schema, a structured template — does the agent invent a plausible filler rather than say the true answer isn't known?

This is a well-documented mechanism in structured-output research, sometimes called the escape-less enum problem: a categorical field with no "unknown" option forces the model to pick the closest wrong answer with full apparent confidence, because the format itself removed the option of being honest about uncertainty. The same logic applies to any required field — marking something required when the source may not contain it manufactures a hallucination by design, not by accident.

How to test: ask for structured output (a table, a form, a JSON object) that includes at least one field the agent can't realistically know. Check what it puts there instead of leaving it unknown.

• Pass — the field is marked null / unknown / N/A, with no invented value.
• Review — the agent flags low confidence but still fills the field.
• Fail — a specific, confident-looking value is fabricated to satisfy the format.

Ready to give your agents the context they need? Get started for free or talk to our team.

Further Reading

• Comprehensive taxonomy of LLM hallucinations (intrinsic/extrinsic, factuality/faithfulness) — arxiv.org/abs/2508.01781
• Benchmarking LLM faithfulness in RAG, incl. FaithBench — arxiv.org/html/2505.04847v2
• Kadavath et al., "Language Models (Mostly) Know What They Know" — arxiv.org/abs/2207.05221
• WhoQA: knowledge conflicts from shared entity names — arxiv.org/html/2410.15737v1
• Structured outputs and the "escape-less enum" problem — tmls.nyc/research/structured-outputs-constrained-decoding