Why AI code review hallucinates — and the two gates that fix it
CCA-Audit — open source (MIT) AI code review has a trust problem, and it's not that it misses bugs. It's that it invents them. If you've run an LLM over a diff, you've seen it: a "possible null dereference" on a value that's guarded three lines up. A "SQL injection" your ORM already parameterizes. A "race condition" that can't happen. And then — worse — it confidently rewrites working code to "fix" the thing that was never broken. The real bug, meanwhile, sits quietly in the noise. The problem isn't intelligence. It's that most AI reviewers report their first impression as a verdict. A model reads a diff, pattern-matches "this looks like X," and emits a finding — without ever going back to check whether X is actually reachable in this code. Humans do a second pass ("wait, is price validated upstream?"). Most AI-review pipelines skip it. Here are two gates that add that second pass — and a stress test showing what they catch. Gate 1: verify findings before you fix (anti-hallucination) The idea is simple: no finding is allowed into the fix plan until a separate step re-checks it against the real code. After the auditors produce findings, a verification pass takes each one and asks three questions: Does the issue actually exist at the cited line? Is it in the code that changed, or a pre-existing thing outside the diff? Is the stated impact real, or already mitigated elsewhere — a guard upstream, a value validated before this point, a config defined in another module? The key design choice: bias the verifier toward refuting. A wrongly-confirmed finding causes a needless (sometimes harmful) fix; a wrongly-dropped one is cheap to recover. So when the evidence isn't clear, drop it or escalate to a human — don't fix on a hunch. This one step kills the majority of hallucinated findings, because hallucinations rarely survive contact with "show me the exact line, and prove the impact can occur." Gate 2: prove the fix maps to the finding (anti-regression + provenance) Catching