No Agent Grades Its Own Homework
You ask Claude to review your code. It says "looks good, clean, well factored". Of course it does. It wrote that code five minutes ago. You just asked the author to grade his own paper, and he gave himself an A. Having an AI review code works. But not by asking the one who just wrote it. Quality doesn't come from a smarter model, it comes from an architecture where no role checks itself. The self-preference bias This isn't a hunch, it's measured. A model evaluating its own output rates it higher than others' at equal quality: the self-preference bias , documented by Panickssery and co-authors in 2024, and it's causal, not correlational. The model recognizes its own style and prefers it. In practice that means the naive loop "write, then review what you just wrote" is broken by construction. You don't get a review, you get a justification. The agent already decided its code was good the moment it produced it; asking again only confirms. The blind reviewer So the first rule: the reviewer is never the author. In my config, the review agents run in a clean context . They don't see the implementation prompt, they don't know what constraints the author set, they meet the diff like a colleague on Monday morning. And when the author is a known model, the reviewer is from a different family , to break style recognition. One detail matters as much as the rest: the developer's name never enters the reviewer's prompt. No "this was written by a senior", no "review this model's work". The author's identity is exactly the information that triggers the bias. We take it off the table. No finding without a receipt The second trap is the opposite of the first. An AI reviewer, especially in a clean context, tends to over-flag: it invents problems to look useful, it flags "vulnerabilities" that aren't. A review that cries wolf on every line is no better than a complacent one: either way, you stop listening. Hence the receipt rule. Every finding must cite a file:line and pass a check bef