Did the Model Upgrade Break Your AI Agent?

Nothing happened. That is the strange part. No deploy. No pull request. Nobody touched the prompt. Your agent ran the way it always ran on Friday, and it runs on Monday, and every dashboard is green. Then a ticket comes in about an answer nobody on your team would have written, and you go looking for the change that caused it, and there is no change on your side. There was a model upgrade. It is the only change to your system that you did not make, cannot find in your own git history, and usually cannot roll back on your own schedule. It is also the one most likely to be announced to you as good news. Why a model upgrade does not look like a bug Because it is not one. The new model really is better. Better on reasoning, better on code, better on the evaluations the lab published beside it, and probably better on yours too, if what you measured was the average. Better and same are different words. Your product was not built on the average. It was built on a specific set of behaviours you watched, liked, and then quietly encoded into everything downstream: how long the answers run, how much the thing hedges, which tool it reaches for first, what it does when a request is vague. None of that appears in release notes. All of it can move. And when it moves, nothing throws. There is no stack trace for "this answer is now worse in a way a customer will notice." Your tests keep passing, because your tests check that the JSON parses and the fields are there, and the JSON still parses and the fields are still there. The three things that actually move Shape. Answers get longer, or shorter, or start opening with a summary they never used to open with. Harmless, right up until something downstream was written against the old shape. Tool choice. The agent develops a new favourite first move. It takes six calls to do what used to take three, or it stops calling the tool you built for it because it has decided it can answer from memory. This one usually reaches the bill before it