Four ways a baseline quietly destroys the anomaly detector built on it

Every anomaly detector answers one question: compared to what? That comparison, the baseline, is where I lost the most time on this project, and every failure had the same signature. Nothing errored. No test went red. The numbers stayed plausible. The detector just quietly stopped detecting. Four of them, in the order I found them. 1. The peer group contained the client it was judging Cold-start clients have no history, so they're compared against a pool of other clients' recent benign windows. Reasonable. The pool was keyed by feature: private readonly peer = new Map < FeatureKey , number [] > (); Every benign window every client produced went into the pool that client was later compared against. Including itself. So a client could define its own normality . Feed in enough windows and any behaviour becomes unremarkable — which is precisely the cold-start attacker the layer exists to catch. What made me look was not reasoning, it was an experiment that wouldn't sit still. I was trying to build a demo client that reliably landed in the middle of the response ladder, and holding the traffic shape fixed while changing only the request interval flipped the outcome between allow and step_up : gap=500ms origins=5 → allow (peak 0) gap=700ms origins=5 → step_up (peak 83) gap=800ms origins=5 → allow (peak 0) A knife edge like that is never a tuning problem. The outcome depended on a race between a client's own samples reaching the pool and the pool being consulted. Fix: key the pool per client, and exclude the client under evaluation. for ( const [ clientId , values ] of byClient ) { if ( clientId === excludeClientId ) continue ; // this is what "peer" means … } Afterwards the behaviour became monotone in the actual evidence, and identical at every request interval: origins 1 3 4 5 6 peak score 17 35 59 83 100 tier allow log throttle step_up deny Lesson: if a parameter that shouldn't matter changes the outcome, stop tuning and go find the defect. Knife edges are symptoms. 2.