Lesson 2.3 · D2 Output Evaluation & Validation · 6 min
Fact-check and validate outputs
Validation means independent checks: primary sources, recalculation and traceable evidence.
What the exam tests
Choosing validation steps that actually test accuracy, and rejecting ones that only feel reassuring.
Key ideas
- Go to primary sources
- Official legislation databases, regulator sites, the original study, the publisher's catalogue, the competitor's own documentation.
- Triangulate key inputs
- Check important assumptions against independent data, such as last year's actuals, benchmarks or expert review.
- Make outputs traceable
- Ask Claude to quote the supporting passage, give the clause number or list the record IDs it counted, then spot-check those.
- Know what isn't verification
- Asking “are you sure?”, Claude's stated confidence, two chats agreeing, or regenerating until answers match. None of these test the facts.
- Current facts need live sources
- Prices, rates, regulations and recent events may have changed after Claude's knowledge cutoff. Use web search or official sites.
- Scale effort to stakes
- Light checks for internal brainstorming; rigorous checks for anything public, contractual, financial or safety-related.
Worked example
Claude reports that 35% of 200 support tickets concern login problems.
Weaker
Ask Claude whether 35% is right.
Stronger
Ask Claude to list the ticket IDs it counted, spot-check a sample, and confirm the total.
Why: A traceable count can be checked quickly; a self-assessment can't.
Common traps
- Treating self-review as independent verification (it's a useful first pass, no more).
- Accepting a citation because the source is reputable, without finding the claim in it.
- Using a figure because it “matches what we remember”.
Remember
Verification is independent: source, recalculation or traceable evidence.