The ~+9.4% You Can't Afford to Verify: Evaluating SDAR (and the FinOps of Trying)

Recap. Part 1 framed the supervision problem. Part 2 architected the four-model system on AWS and counted the hardware. Part 3 put the gate on the page in fifteen lines of PyTorch. Now the question that decides whether any of it mattered: does the gate actually earn its keep - and what would it cost to know? This is the close. A verification design, the stability instrumentation that separates a real evaluation from a lucky one, and the FinOps reality that is the reason this whole series is a blueprint instead of a benchmark. SDAR makes two claims, not one It's tempting to reduce the paper to a single number. The reported gains over plain GRPO are real and worth stating - their numbers, not mine : roughly +9.4% on ALFWorld, +7.0% on Search-QA, +10.2% on WebShop accuracy. But the headline percentage is the less interesting claim. The one that matters is the second: SDAR avoids the training instability that naive GRPO+OPSD falls into . The whole point of the gate Part 3 was to keep the teacher's noisy rejections from destabilizing training. So a verification that only measures final task success has tested half the paper. You have to measure stability too - and most reproductions don't. That two-pronged claim dictates the experiment. The verification: three arms, not two You cannot prove SDAR's thesis with a before/after. You need three training runs, identical except for the supervision: Arm What it is What it proves A. GRPO Plain agentic RL, no teacher The baseline the +9.4% is measured against B. Naive GRPO+OPSD Teacher distillation, no gate The instability SDAR claims to fix C. SDAR Gated distillation Part 3 Beats A on score and beats B on stability Drop arm B and you can show SDAR beats GRPO, but you've quietly deleted the paper's actual contribution - there's no evidence the gate did anything a plain teacher wouldn't. Arm B is the expensive arm nobody wants to run and the one that makes the result credible. Metrics that actually test the claim Metric What it mea