My benchmark's Python column was N/A for a year — CPython's 4300-digit limit, and eight other bugs
Submission for DEV's Summer Bug Smash — Clear the Lineup track. The codebase a2a-benchmark is my multi-language A2A (Agent-to-Agent) performance suite: four agents — Python and Go behind Gemini tool-calling via ADK, Node.js and Rust as direct handlers — each compute Mersenne primes with the Lucas–Lehmer test, while a harness sweeps N=1–24 and charts calculation time and round-trip time. The committed results stopped at N=22. I never questioned that. I should have. The headline bug: a whole column of data didn't exist CPython 3.11+ limits int→str conversion to 4,300 digits by default (a DoS mitigation). My Python agent stringified each prime it found: mersenne_primes . append ( str (( 1 << p ) - 1 )) The 24th Mersenne exponent is p=19937, and 2^19937−1 has 6,002 digits . So for any request of 24+ primes, the tool raised ValueError — and the A2A response dutifully delivered the stack-trace text instead of a result: "text": "Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit" The benchmark's Python column was structurally incapable of producing data at N≥24. The kicker: the stringified list was never used . The tool returns only elapsed_time . The fix is deleting the str() — which also removes formatting work from the timed region that the Node and Rust agents never paid (Go had the same dead val.String() call). Fix: PR #1 — plus a switch from time.time() (wall clock, non-monotonic) to time.perf_counter() , and a regression test at count=24. Before/after, N=24 row: Node.js Rust Go Python before 1633.01 ms 812.57 ms 1451.49 ms N/A (crash) after 1616.13 ms 824.10 ms 1531.10 ms 2425.9 ms The harness was reading its data from LLM prose The Python agent's timing came back in two places: a structured elapsed_time in the tool artifact, and the model's prose. The harness regexed the prose first : m = re . search ( r " It took ([\d\.\-e]+) seconds " , text ) In live runs, Gemini said "Calculating the first 5 Mer