今日已更新 80 条资讯 | 累计 20052 条内容
关于我们

I pointed my code reviewer at its own verifier. It found two ways to lie.

seamstressdev 2026年07月07日 02:15 2 次阅读 来源:Dev.to

I built SeamStress. It's a code reviewer with one rule: it only reports what it can prove against your actual code, quoting the exact lines. If it can't prove it, the finding gets demoted to a judgment call. Not presented as fact. That rule is enforced by one small piece of code: the verification gate. It decides whether a finding may be shown as verified_real. Every other part of the tool can be wrong and the damage is bounded. If the gate is wrong, the tool shows you a confident claim it never earned, with a proof label on it, and it renders as success. Silently. So before making the repo public, I ran the tool on the gate. Same pipeline it runs on anyone's code: three blind critics, then synthesis, then per finding verification. Eight model calls. It found two critical defects in its own foundation. Defect one: verified with no evidence behind it The status authority looked like this: const result = verifications . find (( v ) => v . findingId === finding . id ); return result ? result . status : " unverified " ; It trusted the verdict on a finding ID match. It never looked at the evidence. And the schema allowed an empty evidence array and an empty quoted code string. So a result shaped like {status: "verified_real", evidence: []} validated cleanly and certified a finding as proven. The report renderer would put that finding in the headline, under copy promising the exact lines quoted as proof, with nothing attached. The evidence block suppressed the display of the missing proof. It did not remove the finding from the verified set. The fix lives at the authority, not just the schema: if ( ! result ) return " unverified " ; const hasRealEvidence = result . evidence . some (( e ) => e . quotedCode . trim (). length > 0 ); return hasRealEvidence ? result . status : " unverified " ; A verdict is honored only when at least one non empty quote backs it. Checking at the authority also catches the whitespace quote variant that a naive schema minimum would miss. Fixed in

本文内容来源于互联网,版权归原作者所有
查看原文