Why did my benchmark stop at N=22? A debugging story in nine bugs
Submission for DEV's Summer Bug Smash — Smash Stories track. There was a file in my repo called run_benchmark_1_22.py . Not 1 to 24, which is what the harness was written to do. Not 1 to 26, which is how many Mersenne exponents the agents know. Twenty-two. A chart in the README — a2a_latency_times_1_22.png — agreed. At some point, past-me had decided the benchmark ends at 22, committed the evidence, and moved on. This summer, hunting for a Bug Smash target, I finally asked: why 22? The setup a2a-benchmark compares A2A agent performance across four languages. Python and Go sit behind Gemini tool-calling (ADK); Node and Rust are bare HTTP handlers. Each computes Mersenne primes with Lucas–Lehmer; a harness sweeps N from 1 to 24 and draws two charts. I ran the full sweep. At N=24, the Python column printed N/A . Every other language returned data. There it was — not a decision, a crash , worked around by shortening the run until it stopped hurting. The 4,300-digit wall The Python agent's response at N=24 wasn't even subtle about it: "Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit" CPython 3.11 added a default cap on int→str conversion — 4,300 digits — as a denial-of-service mitigation. My agent stringified every prime it found. The 24th Mersenne prime, 2^19937−1, has 6,002 digits . Here's the part that made me laugh out loud: the stringified list was never returned . The tool reports only its elapsed time. The line that had silently amputated my benchmark at N=23 was decorative. The fix was git rm energy: delete the str() , keep the raw int. Go had the identical dead weight ( val.String() ) inside its timed region — it just happened not to crash. One deleted expression, and a column of data that had never existed came into being: N=24, Python, 2,425.9 ms. It gets worse before it gets better With the agents finally running, I kept pulling the thread. The harness parsed Python's elapsed time out of th