今日已更新 339 条资讯 | 累计 19899 条内容
关于我们

Checkpoint-Skip Gate: Task Success 100%, Checkpoint Never Ran

Alexey Spinov 2026年07月12日 17:35 4 次阅读 来源:Dev.to

Checkpoint-skip gate: a multi-agent pipeline can finish with task_success: true while the mandatory confirmation checkpoint never ran. checkpoint_skip_gate.py replays a recorded JSONL trajectory against a declarative spec of mandatory checkpoints and handoff contracts, offline, and blocks when the road was wrong. The verdict never consults the final metric. That is the point. AI disclosure: I wrote checkpoint_skip_gate.py with an AI assistant and ran it myself, offline, on Python 3.13.5, standard library only, no network. Every number, exit code, and hash in the output blocks below is pasted from a real local run. I ran each scenario twice to confirm STDOUT is byte-for-byte identical, and the tool prints a sha256 of its own report so you can reproduce the exact bytes. The Alberta write-up and the arXiv paper I cite are other people's work, attributed inline, and their numbers stay out of my fixtures. In short: task_success=true proves the pipeline arrived. It does not prove the mandatory steps happened, happened in order, or that each agent-to-agent handoff delivered what the next agent assumed. A trajectory can be perfectly green and structurally wrong. The gate replays a recorded trajectory against a spec you declare: checkpoints that must precede specific actions, plus contracts for each handoff (required fields, verified flags). The final metric is printed for contrast and ignored for the verdict. The demo that matters: two trajectories identical except one JSONL line, the confirm_with_user checkpoint event. Both end task_success: true . Delete that line and the verdict flips from PASS exit 0 to BLOCK exit 1 checkpoint-skipped . It also tracks unverified values across handoffs. A number that travelled a connected chain of two handoffs with no hop verifying it blocks as unverified-claim-propagated-2-hops . Everyone shared the number. Nobody verified it. Offline, keyless, zero network, fail-closed: broken input exits 2, never a silent green. The whole 8-fixture sw

本文内容来源于互联网,版权归原作者所有
查看原文