Trace numbers to source

The most critical QA check: every number in a deliverable must be traceable to a real source. Not "probably right". Not "Claude said so". Actually traceable. This lesson is the routine.

Why this matters

Claude can hallucinate numbers. Not often, but often enough that uncaught fabrications destroy trust. A client catches one fake number in a report and assumes the rest of the report is suspect too.

Reputation is binary on this. You're either reliable with data, or you're not trustworthy.

The trace-every-number command

After Claude builds any deliverable with numbers

For every metric in this deliverable, show me the source. Which database query, MCP pull, or file did each number come from? Go line by line - don't summarise.

Claude walks through each number. Every one should trace to:

• A specific database query (with the SQL shown)
• A specific MCP call (with the parameters)
• A file you provided (with the file path + cell/row reference)

Anything that traces to "Claude calculated" or "based on the analysis" is suspect. Push back.

Spot-check specific numbers

You don't have time to trace every number every time. Spot-check instead:

1. Pick 3-5 numbers from the deliverable - ideally including any number that seems surprising or important
2. Ask Claude to show you exactly how each was calculated
3. Cross-reference one or two against the source platform (open Google Ads + check the ROAS number)
4. If all 3-5 spot-checks pass, you can trust the rest with moderate confidence
5. If even 1 doesn't match, do a full trace on everything

Where did the $42,000 revenue figure in section 3 come from? Show me the raw data + the exact query.

Spot-check: the 3.2x ROAS for Meta this month. How was it calculated?

For every metric in the top section, name the source + date range.

Check date ranges

Classic mistake: Claude pulls data for the wrong period. Last 7 days vs last 30 days. Calendar month vs trailing 30. This year vs last year.

Confirm the date range for all data in this report. Is everything from [period I expected]? Show me the dates used in each query.

Date-range errors are common + hard to spot in the final output. Always confirm.

Check comparison periods

If the report compares periods (vs prior month, vs YoY), verify both:

The "vs previous period" numbers - what period is Claude comparing to? Is it the right comparison? Show me both periods' raw data.

Common failure: comparing last 30 days vs the 30 days BEFORE that, when you wanted last 30 days vs same 30 days last year.

Red flags in numbers

Even without formal tracing, watch for

• Suspiciously round numbers - "exactly 1,000 conversions" - real data is rarely this clean
• Percentages that don't add to 100% - segments of a whole
• ROAS / CPA / CPM that don't match math - ROAS = revenue / spend. If those 3 numbers don't compute, one is wrong.
• Revenue that's orders of magnitude off - $42K vs $4.2K typo
• "N/A" or "undefined" in fields that should have data
• Identical numbers across different metrics - lazy hallucination

Any of these = trace the specific number before approving.

Enforce anti-fabrication as a rule

Rather than waiting for review to catch hallucinated numbers, bake the rule into your setup. Add to your global CLAUDE.md: "NEVER fabricate metrics. Every number in any deliverable must trace to a verifiable source." Or build a skill for it (Module 09 Lesson 5) if you want consistent enforcement across sessions.

Effect: Claude enforces this at the generation stage

• Every metric Claude produces must have a traceable source in context
• Claude will refuse to generate numbers it can't source
• Uses clearly marked placeholders when data is unavailable

Catches hallucinations before they reach your review. Much cheaper than catching them after.

What to do when you find a wrong number

1. Confirm with the source - is the number actually wrong, or was your initial source mismatched?
2. If confirmed wrong: tell Claude "the [X] number is wrong. Correct it to [Y]. Re-check everything downstream."
3. Claude re-verifies the cascading numbers and fix es the report
4. Update the relevant rule or skill: "for reports, always trace each metric to source before finalising"
5. Add to CLAUDE.md if it's a pattern that applies across projects

Power-user tips

• Check date ranges first - most common failure, easiest to spot
• Use CAPS in safety rules - "NEVER fabricate metrics" improves adherence
• Bake anti-fabrication into CLAUDE.md - prevention is cheaper than cure. Install the /anti-fabrication skill from Module 09 Lesson 5.
• Build your own verification skill - if you trace numbers on every report, turn it into a skill via /skill-creator (Module 09 Lesson 3). One command walks through the full trace.
• Demand specificity - "the number came from the ads database" is not enough. Need the exact query + date range. Action items

☐ Add to your global CLAUDE.md: "NEVER fabricate metrics - trace every number to source"

☐ Install the /anti-fabrication skill from Module 09 Lesson 5 for generation-stage enforcement

☐ On every report, run the "trace every metric" command before review

☐ Always spot-check 3-5 specific numbers against the source platform

☐ Always verify date ranges + comparison periods explicitly

Next lesson: QA checklists per deliverable type.

Exercises

1. Review the concepts covered in this lesson: Trace numbers to source.
2. Write down your key takeaway from this lesson.
3. Practice running any commands or prompts mentioned above inside your terminal.