Your Guardrails Are a Firewall. Your Failures Are a Cascade

TL;DR— Most production AI teams build safety layers using the content-moderation mental model: classify input, classify output, block or pass. But the incidents that actually take down AI systems in production look like distributed-systems failures— retries amplifying bad state, cascading errors across agent steps, silent drift with no rollback path. Guardrails need to borrow from SRE, not from trust-and-safety. Ask a team how they handle AI safety in production and you'll get the same answer almost every time: an input classifier, an output classifier, maybe a moderation API bolted on the side. This is the content-moderation mental model— filter bad stuff in, filter bad stuff out. It's borrowed wholesale from trust-and-safety teams who spent a decade building spam filters and abuse detectors. It's also the wrong model for most of what actually breaks AI systems in production. The incidents that page you at 2am rarely look like a jailbreak slipping past a classifier. They look like distributed-systems failures: a retry loop that amplifies a bad tool call, a hallucinated intermediate result that poisons every downstream step, a silent shift in output distribution that nobody notices until a customer complains three weeks later. These are not content problems. They're systems problems, and they need systems solutions. The Cascade, Not the Jailbreak Consider a typical agent pipeline: retrieve context, call a model to plan, call tools, call a model again to synthesize, maybe loop if a tool fails. Each step has some non-zero error rate. In a single-call chatbot, that error rate is the whole risk surface. In a five-step agent chain, errors compound, and worse, they compound non-linearly because failed steps often trigger retries, and retries on a stateful action are not free. A model that hallucinates a tool argument doesn't just produce one bad output— it produces a bad state that the next step reasons over as if it were true. If that next step is another LLM call, it wi