Origin Part 19: The Number Was Wrong

The brain layer was scoring high because the test was leaking. The actual capability was being silently rejected by a misconfigured gate. Both findings landed in the same week. Part 18 ended on a clean diagnosis. The brain layer reasoned correctly when the encoder fed it correct inputs. The encoder didn't always feed it correct inputs. So the path forward was upstream: more physics-shaped training data for the encoder, retrain, re-validate. I wrote the drops, kicked off the retrain, and watched the held-out eval climb. It hit twenty-three out of twenty-six. Eighty-eight percent. The number I'd been chasing. I sat with that for an evening. Twenty-three of twenty-six on compositional reasoning probes the model had never seen during training. The Phase 8 cutover gate from Stage D had been sixty percent. I was thirty points past it. The brain layer had not only survived its missing-from-production months, it had come back stronger. The number was wrong. I figured this out the next morning while writing what was going to be the celebration commit. Something nagged about the eval set. The training data generator built the eval pairs independently from the training pairs, drawn from a different source list. That should have given me a clean train/test split. But I noticed the eval generator was running before the training generator wrote its file, and neither side knew about the other. I dropped into a Python shell and intersected the two pair sets by their input-output keys. Twenty-three of twenty-six held-out probes were also present in training data. Eighty-eight percent of my held-out eval wasn't held out. The model wasn't generalizing. It was memorizing the answers it had already been shown, then being graded on whether it remembered them. The three pairs that were genuinely unseen, I checked those separately. The model got one right. Three out of twelve when I went back through other historical evals and ran the same overlap check. About a quarter, with no statistica