Opinion: AI Server Changes Need a Fault Drill, Not Just a Rollback Plan

A rollback plan tells you how to undo an AI change, but not what breaks first when the change stays in place. Most production incidents do not begin with a deliberate rollback; they begin with an unexpected failure mode that the author never tested. I now treat a passing fault drill as a precondition for reviewing any AI-generated server patch. The drill runs on a disposable server before a human reads a single line of the diff. Why a rollback plan is not enough A rollback plan answers a question about the past: how do we return the system to a known state? A fault drill answers a question about the future: what happens when this change meets a condition the author did not imagine? The second question decides whether you get paged at 3 a.m. A change with a perfect rollback can still fail in a way that nobody notices until the data is gone. Free model access changes the economics of this argument, because generation stops being the bottleneck and verification starts. When a draft is nearly free, the cheapest verification is the one that breaks the change on purpose. A rollback plan is documentation; a fault drill is evidence. Documentation tells you what should happen, while evidence tells you what actually happens on a real service manager. The fault drill in five steps The workflow assumes two cheap resources: a model that generates failure hypotheses from a diff, and a server that can be destroyed after the drill. MonkeyCode's free model access covers the first, and its free server option covers the second, so a drill costs almost nothing. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Any ephemeral VM or container host works if you prefer a different provider. 1. Generate failure modes before you apply anything Ask the model to enumerate failure modes for the diff, and forbid it from proposing fixes, because fixes are a distraction at this stage. The prompt below is the one I use, and it produces a catalog that the drill can test.