I Round-Tripped 2,249 Test Fixtures Through sqlfluff's Auto-Fixer. Eight Came Back Unparsable.

Auto-fix is the most dangerous feature a linter has. Every other part of the tool only reads your code. --fix writes to it, usually across a whole repository, usually in CI, usually without anyone reading the diff. So it is worth asking a blunt question: does the fixer ever produce output the tool itself cannot read back? There is a cheap way to find out, and the input you need is already sitting in the repository. The invariant If a fixture parses cleanly, its fix output must parse cleanly too. That is the whole idea. One line. No fuzzer, no grammar model, no corpus to build. It works because a linter's test fixtures are the best adversarial input anyone will ever hand you for free. They were written by the maintainers, deliberately, to sit on the edges of the grammar: every operator, every dialect quirk, every syntax someone once filed a bug about. Nobody could commission a better corpus, and it ships with the source. The fixer, meanwhile, has usually never been pointed at all of it at once. Fixtures get used to test the parser . Running them through the fixer is a different question, and mostly nobody asked it. The run sqlfluff, at a54076a . 2,249 fixtures, 28 dialects, roughly an hour single-process. For each fixture: parse it. If it parses, fix it, then parse the output. Classify the result: CORRUPTION — parsed before, does not parse after. UNSTABLE — fixing twice does not converge. FIX_CRASH — the fixer threw. Raw result: 8 CORRUPTION, 31 UNSTABLE. Thirty-nine hits. Thirty-seven of them were not bugs. Triage is the entire job Finding 39 things is easy. Knowing which 2 are real is the skill, and getting it wrong in public is how you teach a maintainer to ignore you. UNSTABLE is almost always noise All 31, in this run. sqlfluff's CLI loops the fixer until output stabilises, so a single pass that has not yet converged is expected behaviour , not a defect. Reporting those would have been 31 false positives in one issue. Nine of my first ten findings on an earlier