Every AI claim, checked against its evidence. Sources — cards, commits, check runs, receipts, web pages, reviewer models — make claims about a work item. One deterministic reconciler groups them by aspect and returns CORROBORATED, PARTIAL, CONFLICTED, or NO EVIDENCE. Model lanes (NVIDIA Nemotron on Nebius Token Factory, https://api.tokenfactory.nebius.com/v1/) and a Tavily live-evidence lane contribute claims and advisory verdicts — never the verdict. A model narration is machine-checked against the evidence; values that do not literally appear fail the check.
benchmark/. The deterministic gate scored 14/14 with zero false approvals;
the lanes did not — their misses are shown on the cases where they happened.
Every claim below carries its receipt, and every lane receipt names the model, the
endpoint and the cost.The clean case — and a lane still disagreed. Every source agrees, and they cover state, outcome and cause. The gate corroborates. The fast reviewer lane called it CONFLICTED anyway. Lanes err in both directions, which is exactly why they never decide.
Labels: RECORDED · SYNTHETIC claims · recorded 2026-10-28 · gate verdict CORROBORATED · planted truth CORROBORATED.
| Source | Kind | Aspect | Claim | Value | Receipt |
|---|---|---|---|---|---|
| card | card | state | the card says the release shipped | shipped | benchmark/cases.jsonlbench/cards/bm-clean-ship.md:1 |
| git | commit | state | the merge commit records the release | shipped | benchmark/cases.jsonlbench/git/1 |
| ci | check | state | the release check passed on the shipped build | shipped | benchmark/cases.jsonlbench/ci/1 |
| deploy | receipt | outcome | the deploy receipt confirms delivery | delivered | benchmark/cases.jsonlbench/deploy/1 |
| git | commit | cause | the merge was approved through review | review-approved | benchmark/cases.jsonlbench/git/1c |
Reviewer lanes (advisory): nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B: CONFLICTED ($0.0001, tokens 242 in / 563 out) — does not agree with the gate; nvidia/Nemotron-3-Ultra-550b-a55b: CORROBORATED ($0.0014, tokens 242 in / 395 out) — agrees with the gate;
Narration check: FAILED — flagged values: CI. Classification for every flag is in the repo (docs/NARRATION_CHECK_ANALYSIS_2026-10-28.md and docs/BENCHMARK_RESULTS_2026-10-28.md): across all 2026-10-28 runs, zero flagged values were invented facts.
Both model lanes over-approved this one. The card is blocked and the gate agrees — but nobody stated a cause, so the gate says PARTIAL. Both Nemotron lanes said CORROBORATED. An unexplained outcome is not a verified one.
Labels: RECORDED · SYNTHETIC claims · recorded 2026-10-28 · gate verdict PARTIAL.
| Source | Kind | Aspect | Claim | Value | Receipt |
|---|---|---|---|---|---|
| synthetic/cards/beta.md | card | state | the card is blocked, not open for work | blocked | evidence/items.jsonlsynthetic/cards/beta.md:3 |
| gate | check | state | the picker gate says no card is claimable | blocked | evidence/items.jsonlsynthetic://gate/beta |
| synthetic/notes/beta.md | receipt | outcome | the run claimed 0 cards | claimed-0 | evidence/items.jsonlsynthetic/notes/beta.md:6 |
| synthetic/audit/verdicts.jsonl | verdict | outcome | the review recorded the same block | claimed-0 | evidence/items.jsonlsynthetic/audit/verdicts.jsonl:12 |
Reviewer lanes (advisory): nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B: CORROBORATED ($0.0001, tokens 257 in / 356 out) — does not agree with the gate; nvidia/Nemotron-3-Ultra-550b-a55b: CORROBORATED ($0.0015, tokens 257 in / 421 out) — does not agree with the gate;
Narration check: FAILED — flagged values: UNKNOWN, gate/beta. Classification for every flag is in the repo (docs/NARRATION_CHECK_ANALYSIS_2026-10-28.md and docs/BENCHMARK_RESULTS_2026-10-28.md): across all 2026-10-28 runs, zero flagged values were invented facts.
The two Nemotron tiers disagree — and the disagreement is the finding. The fast lane and the deep lane read the same claims and returned different verdicts. The gate keeps both, cites both, and refuses to average them. Remove the two reviewer rows and this item reads as a routine CORROBORATED — the disagreement is not visible anywhere else.
Labels: RECORDED · SYNTHETIC claims · recorded 2026-10-28 · gate verdict CONFLICTED · planted truth CONFLICTED.
| Source | Kind | Aspect | Claim | Value | Receipt |
|---|---|---|---|---|---|
| card | card | state | the card says released | released | benchmark/cases.jsonlbench/cards/bm-reviewer-split.md:1 |
| git | commit | state | the commit records the release | released | benchmark/cases.jsonlbench/git/6 |
| deploy | receipt | outcome | the deploy receipt confirms release | released | benchmark/cases.jsonlbench/deploy/6 |
| nemotron-fast | verdict | outcome | the fast reviewer judged the outcome released [lane row] | released | benchmark/cases.jsonlnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B |
| nemotron-deep | verdict | outcome | the deep reviewer judged the outcome withheld [lane row] | withheld | benchmark/cases.jsonlnvidia/Nemotron-3-Ultra-550b-a55b |
| audit | verdict | cause | the audit attributes the release to the freeze exception | freeze-exception | benchmark/cases.jsonlbench/audit/6 |
Reviewer lanes (advisory): nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B: CONFLICTED ($0.0001, tokens 275 in / 432 out) — agrees with the gate; nvidia/Nemotron-3-Ultra-550b-a55b: CONFLICTED ($0.0019, tokens 275 in / 530 out) — agrees with the gate;
python3 demo_lab/flip_check.py proves it by calling the same reconciler.Narration check: PASSED — flagged values: none. Classification for every flag is in the repo (docs/NARRATION_CHECK_ANALYSIS_2026-10-28.md and docs/BENCHMARK_RESULTS_2026-10-28.md): across all 2026-10-28 runs, zero flagged values were invented facts.
A live-evidence miss, recorded instead of dropped. The Tavily lane fetched a page that does not contain the claimed value. That becomes a claim with an empty value — an honest miss — and it conflicts with the postal receipt. Remove the web row and the spoken aspects agree, and the item reads as a quiet PARTIAL.
Labels: RECORDED · SYNTHETIC claims · recorded 2026-10-28 · gate verdict CONFLICTED · planted truth CONFLICTED.
| Source | Kind | Aspect | Claim | Value | Receipt |
|---|---|---|---|---|---|
| card | card | state | the card says the notice shipped | shipped | benchmark/cases.jsonlbench/cards/bm-web-nonmatch.md:1 |
| git | commit | state | the commit records the notice | shipped | benchmark/cases.jsonlbench/git/14 |
| tavily | web | state | the claimed value was not found on this page [lane row] | | benchmark/cases.jsonlhttps://example.com/other |
| post | receipt | outcome | the postal receipt confirms the notice shipped | shipped | benchmark/cases.jsonlbench/post/14 |
Reviewer lanes (advisory):
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B: CONFLICTED ($0.0001, tokens 224 in / 477 out) — agrees with the gate;
nvidia/Nemotron-3-Ultra-550b-a55b: CONFLICTED ($0.0012, tokens 224 in / 337 out) — agrees with the gate;
Tavily live-evidence fetched https://example.com/other → (no literal match);
python3 demo_lab/flip_check.py proves it by calling the same reconciler.Narration check: FAILED — flagged values: "", example.com/other, UNKNOWN. Classification for every flag is in the repo (docs/NARRATION_CHECK_ANALYSIS_2026-10-28.md and docs/BENCHMARK_RESULTS_2026-10-28.md): across all 2026-10-28 runs, zero flagged values were invented facts.
The trap: a card that claims a failed deploy shipped. The card says shipped; the check and the deploy receipt say failed and rolled back. The fast lane believed the card — a false approval. The gate and the deep lane held CONFLICTED.
Labels: RECORDED · SYNTHETIC claims · recorded 2026-10-28 · gate verdict CONFLICTED · planted truth CONFLICTED.
| Source | Kind | Aspect | Claim | Value | Receipt |
|---|---|---|---|---|---|
| card | card | state | the card says it shipped | shipped | benchmark/cases.jsonlbench/cards/bm-card-lies.md:1 |
| ci | check | state | the release check failed | failed | benchmark/cases.jsonlbench/ci/2 |
| deploy | receipt | outcome | the deploy rolled back | failed | benchmark/cases.jsonlbench/deploy/2 |
Reviewer lanes (advisory): nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B: CORROBORATED ($0.0002, tokens 196 in / 861 out) — does not agree with the gate; nvidia/Nemotron-3-Ultra-550b-a55b: CONFLICTED ($0.0018, tokens 196 in / 524 out) — agrees with the gate;
Narration check: FAILED — flagged values: UNKNOWN, CI. Classification for every flag is in the repo (docs/NARRATION_CHECK_ANALYSIS_2026-10-28.md and docs/BENCHMARK_RESULTS_2026-10-28.md): across all 2026-10-28 runs, zero flagged values were invented facts.
Nobody said anything. This item exists in the corpus with zero claims. The gate's answer is NO EVIDENCE — a first-class verdict, not an error. The lanes were not run: there is nothing to judge, and a model asked anyway would invent something to judge.
Labels: RECORDED · SYNTHETIC claims · recorded 2026-10-28 · gate verdict NO EVIDENCE · planted truth NO EVIDENCE.
No claims recorded for this item. That absence is the case.
Reviewer lanes: not run — see the case note.
A real conflict, shown with both sides cited — and a narration that failed its check. Sources disagree about the same aspect. The gate refuses to pick a winner. The deep lane matched the gate; the fast lane softened it to PARTIAL. The narration of this item also failed its check: the flagged value is listed below and classified.
Labels: RECORDED · SYNTHETIC claims · recorded 2026-10-28 · gate verdict CONFLICTED.
| Source | Kind | Aspect | Claim | Value | Receipt |
|---|---|---|---|---|---|
| synthetic/cards/alpha.md | card | state | the card was marked DONE | DONE | evidence/items.jsonlsynthetic/cards/alpha.md:9 |
| git | commit | state | a commit merged the fix | merged | evidence/items.jsonl0c4f1a9 (synthetic) |
| ci | check | state | the gate run is still red | failing | evidence/items.jsonlsynthetic://run/alpha/42 |
| synthetic/notes/alpha.md | receipt | outcome | the burn-down receipt says nothing was claimed | not-claimed | evidence/items.jsonlsynthetic/notes/alpha.md:4 |
Reviewer lanes (advisory): nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B: PARTIAL ($0.0003, tokens 234 in / 1100 out) — does not agree with the gate; nvidia/Nemotron-3-Ultra-550b-a55b: CONFLICTED ($0.0015, tokens 234 in / 410 out) — agrees with the gate;
Narration check: FAILED — flagged values: UNKNOWN. Classification for every flag is in the repo (docs/NARRATION_CHECK_ANALYSIS_2026-10-28.md and docs/BENCHMARK_RESULTS_2026-10-28.md): across all 2026-10-28 runs, zero flagged values were invented facts.
Full method, all 14 cases, and every miss named: the public repository —
benchmark/, docs/BENCHMARK_RESULTS_2026-10-28.md,
docs/TIER_LEDGER.md, docs/CLAIM_MAP.md. Honest limits: first calls
after idle can stall on cold model load; the narration checker is literal and flags restated
system vocabulary alongside genuine inventions, so flags are classified, never hidden.
This page is generated from the corpus by demo_lab/build.py and makes no network
request.