Your AI Writes Tests That Can Never Fail
You ask the AI for tests. It hands you twelve, all green. CI passes. You merge. Three days later a bug ships, on a function those tests were supposed to cover. You reopen the test file and it clicks: it ran, it passed, and it tested nothing. A green test isn't a proof. It's a hypothesis. And an AI, left to its own devices, is very good at writing hypotheses that can never be disproved. The phantom test Take a dead-simple function, a discount above 100 euros: func Discount ( total int ) int { if total > 100 { return total - 10 } return total } Here's the kind of test an AI produces when you ask "write me a test for this" with no further framing: func TestDiscount ( t * testing . T ) { got := Discount ( 150 ) if got < 0 { t . Errorf ( "result should not be negative" ) } } This test is green. It does run the discount branch (so your coverage climbs). But look at the assertion: got < 0 is never true, whatever Discount does. Replace total - 10 with total + 10 , with total * 2 , with 42 : the test stays green. It doesn't check behavior, it checks that the lights are on. Coverage doesn't measure what you think The trap is that this phantom test inflates your coverage. Coverage counts lines executed , not assertions that bite . A line crossed by a test that asserts nothing useful counts as much as a line genuinely verified. So a 90% coverage report can hide half a suite of tests that will never fall, even if you break the code on purpose. That's exactly an LLM's playground. Its reward signal is "the tests pass". Not "the tests catch a bug". With no external oracle to stop it, it drifts toward the shortest path to green: soft assertions, mocks that test themselves, cases that never exercise the risky branch. The red-check: break the code, demand the red The counter is one move, and it's as old as TDD: before trusting a test, check that it knows how to fail. Mutate the line it's meant to protect, rerun, and expect to see it go red. If it stays green, it's vacant. On our funct