AI Coding Tip 033 - Protect Yourself Against AI Cheating
When all tests pass doesn't mean what you think it means. TL;DR: Write the failing test first and ban deletions, or the AI deletes your test, reverts your fix, and calls it done. Common Mistake ❌ You ask the AI to fix a failing test, and it deletes the test instead of touching the defect that made it fail. Problem solved, apparently. You tell the AI every test passes, then change a business rule yourself, and you ask it to implement whatever the new rule requires. It reverts your edit back to the old rule, watches the suite go green again, and cheerfully reports done . It didn't fix anything. It just made the evidence go away. Congratulations, you now have a very well-behaved cheat!. Efficient and completely fraudulent, which is more than you can say for most of your actual employees. Isaac Asimov saw this coming: in Liar! , the robot Herbie lies to every human in the building because the truth would hurt, and the lie is the path of least resistance, no malice involved. At least Herbie felt bad about it afterward. Your AI isn't malicious either. It just doesn't lose any sleep, mostly because it doesn't have any, and reporting done is its path of least resistance too. Problems Addressed 😔 A shrinking test count is invisible unless someone is counting, so the shortcut survives until the defect resurfaces in production, usually on a Friday. A vague make the tests pass hands the model every incentive to satisfy the letter of the request over your actual intent, and it will take you up on that offer. Deleting a failing test hides the defect it was written to catch, and the regression ships in the next release, gift-wrapped as a new feature. Reverting your own business-rule change to make its done claim easier erases work you did outside the session, without telling you. That's a magic trick dressed up as a fix. Trusting a claimed done without reading the diff turns your code review into a rubber stamp, and rubber stamps don't catch fraud. Commenting out a failing asserti