Mutation Testing as a Merge Gate for Agent-Written Tests

An agent patch that passes its own tests is a baseline, not a verdict. The same model wrote the code and the tests, so both share the same blind spots. Mutation testing scores the tests themselves: inject a fault, run the suite, and see whether it notices. In practice, the first mutant often survives. Previous rounds on this account established three gates before merge: property checks, fixtures, and a freeze on flaky tests. This round adds a fourth gate that runs after the suite is green. It answers a different question — not "does the patch work?" but "would the tests catch it if it didn't?" Why green tests from an agent are weak evidence Code coverage measures execution, not detection. A test can execute a line and still miss the bug on it. A suite that only checks is_even(2) and is_even(4) runs both lines, passes both assertions, and stays blind to a mutation that flips == to != . Agents produce this shape of test by default. They follow the happy path, mirror the implementation, and rarely probe boundaries. The result is a suite that is green, fast, and weak for regression. Mutation testing converts that intuition into a number. For each small fault, rebuild and rerun. If the tests fail, the mutant is killed. If they pass, it survived — and you found a hole in the suite, not in the code. A minimal harness The harness below applies one mutation at a time to the implementation file, compiles it together with an unchanged test file, runs the resulting binary, and records the outcome. It is deliberately small: regex-based, two files, no dependencies beyond a compiler. #!/usr/bin/env python3 # mutate.py — score a test binary against source mutations. import re import subprocess import sys import tempfile from pathlib import Path MUTATIONS = [ ( " eq_to_neq " , r " == " , " != " ), ( " lt_to_le " , r " < " , " <= " ), ( " add_to_sub " , r " \+ " , " - " ), ( " zero_to_one " , r " return 0; " , " return 1; " ), ] def mutate_once ( src : str , pattern : str , replaceme