Zero Is Not a Score

The evals for my agent skills scored 0% for as long as I had records. Not low. Not noisy. Exactly zero, every skill, every run. And I believed it. For months I thought my skills were bad, because the number said so and the number never wavered. Then one night I actually read the harness. It had fallen back to the wrong auth token. Every call it made came back 401, and it quietly graded each one a failure. The skills never got a chance to fail on their own. I was not measuring them at all. I was reading a broken thermometer. Real weakness is jagged Here is what took me too long to see. When a system is genuinely bad, it scores 40% one week and 60% the next. It passes the easy cases and trips over the hard ones. It has good days. Incompetence has texture, because an incompetent system is still in contact with the world, and the world varies. A flat number has no texture. A flat number means the measurement stopped touching the thing being measured somewhere upstream, and what you are reading is the instrument's resting state. Doctors know this. A heart monitor drawing a perfectly straight line does not mean the patient is calm. Only a broken thermometer writes the same number every time. Key insight: A performance number with no variance is a reading of the instrument, not of the thing being measured. The same bug in three industries I run systems in advertising, in healthcare billing, and in agent operations, and the same shape shows up in all of them. In advertising I found a dashboard figure that had been hardcoded for two years. Nobody questioned it, because it looked right, and it looked right because it never moved. In agent operations, an account-rotation bug in one of my pipelines overwrote every real error with the same generic message, "no active accounts," so for a while every distinct failure in that system looked identical. And in the denial-assessment engine I run for a medical-billing operation, an agreement metric came back at 44.7%, alarmingly low, un