Quality Isn't Accidental — Maker/Checker Separation and Automated Validation

The Core Argument : AI agent reliability isn't achieved by "making the agent smarter" — it's achieved by the simple engineering principle of separating validation from generation . Quality isn't accidental. It's designed. What You'll Learn : Maker/Checker separation, 6 termination conditions, and an automated feedback loop — all with runnable code. 0. Prerequisites Python ≥ 3.10 OpenAI API Key (or compatible interface) pip install openai>=1.0.0 (Optional) pip install anthropic>=0.30.0 if using Claude as Checker 1. The Pain: Why "Agent Checks Itself" Is a Trap 1.1 The Copying of Cognitive Bias A team built a data-analysis agent. It pulled sales data from a database and generated business reports. The team added a "self-review" step: after generating, the agent told itself "please check if the data you just output is accurate." Result? The agent always replied "data is accurate." Even when the team deliberately injected obvious errors (e.g., monthly sales of -50M RMB), the agent confidently said everything was fine. This isn't the model being "disobedient." It's a more fundamental issue: when the generator and checker are the same entity, the check is just a restatement of the generation process — not real validation. The checker carries the exact same cognitive bias, knowledge boundaries, and reasoning path as the generator. 1.2 The Amplifier Effect of Confirmation Bias Self-checking also triggers a subtler problem: confirmation bias amplification. The model builds a "belief state" during generation; when re-examining, it tends to confirm rather than overturn. Experiment data (from Anthropic research): Same model does "generate → self-review": ~12% error-correction rate A separate model instance reviews: ~37% error-correction rate A different model family reviews: ~52% error-correction rate 1.3 The Value of Independence First principle of quality assurance: the checker must be independent of the generator. In agent architecture, the engineering expression of this is