Lesson 2.2 · D2 Output Evaluation & Validation · 7 min
Identify hallucinations, inconsistencies and bias
Know the red flags for invented content, internal contradictions and biased framing.
What the exam tests
Recognising which parts of an output most need checking, and what to do with biased or contradictory content.
Key ideas
- Hallucination red flags
- Precise statistics with vague or no sources; quotes attributed to named people; citations to papers, cases or reports; page or section references; claims about recent events; figures for data you never provided.
- Internal inconsistency
- The same figure stated two ways, “three options” introduced but four described, different dates for one event. Resolve these from the source, never by averaging or picking one.
- Bias in content
- Stereotypes about age, gender, ethnicity, income or location; coded hiring language (“young”, “rockstar”, “native speaker”); personas built on assumptions instead of data.
- Bias in selection
- Summaries that include only positive findings, or omit safeguarding concerns or missed targets, are misleading even if every sentence is true.
- What to do
- Verify or remove unsupported claims; rewrite biased language using job-related or evidence-based criteria; restore omitted material findings.
Worked example
A blog draft says “9 out of 10 dentists recommend our toothpaste.”
Weaker
Change it to “8 out of 10” so it sounds less exaggerated.
Stronger
Remove the claim unless the client supplies evidence for it, and check it against advertising rules.
Why: An unsubstantiated statistic is still unsubstantiated after you soften it.
Common traps
- Keeping a dubious citation but making the attribution vaguer (“according to research”).
- Fixing a contradiction by averaging two numbers.
- Treating internal documents as exempt from bias checks.
Remember
Specific + unsourced = verify. Contradictions get resolved from the source.