I spent 11 days optimizing a search ranking that only I could see

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . The symptom: good numbers, no users I publish small automation tools on a marketplace. By August I had 23 of them live. Store search looked fine — measured repeatedly, from a real browser, against the real production endpoint: Search term My rank (store UI, Aug 2) sitemap checker #3 google play audit #1 Real numbers after 89 days: 1 active user across all 23 tools. $0 revenue. A #1 ranking and one user is not a rounding error. It is a contradiction, and I spent a week and a half resolving it in the wrong direction. Eleven days of correct answers to the wrong question If ranking is fine and users are zero, the fault must be downstream — that was the reasoning. So I went looking for it, carefully: Demand analysis. Pulled 3,655 listings, then went deeper to 12,834 to check for sampling bias in the first pass. (There was one. I found it and corrected it.) Naming analysis. Split the corpus by whether the title contained a well-known platform name. Median users: 5 vs 2. Age-cohort analysis. Measured the base rate for new listings: only 11% (n=9) get their first user within 0–3 days of publishing, against 74% at 14–30 days. Mine were young. The zeros were, statistically, unremarkable. Acted on all of it. Renamed 5 tools. Added output schemas across the board — the platform's own quality score went from 74 to 78–79. Every one of those produced a defensible number. Not one of them changed anything. That pattern is the actual signal, and I missed it for too long: when every hypothesis confirms and nothing moves, stop testing hypotheses and start testing the instrument. "It reproduced" is not "it's correct" I had re-measured the ranking several times over those days. Same answer each time. I read that as confirmation. It isn't. Re-running a measurement under identical conditions reproduces the same bias just as faithfully as it reproduces the same truth . Repetition rules out transient noise and