Case Study: A Free Model Wrote a C++ Tree Hasher. The Reference Oracle Found Three Bugs.

Conclusion first: a free model drafted a working C++17 directory hasher in one pass. The draft compiled, ran, and was still wrong. A differential test against standard system tools found three real bugs before the tool ever touched a production cache. Generation was the cheap part. Verification was the deliverable. Background I needed a deterministic hash of a directory tree. The use case was cache invalidation for a small build pipeline: if any file content, name, or symlink target changes, the cache key must change. If nothing changes, the key must stay identical across machines and across checkouts. Hand-writing the tool is maybe 200 lines of std::filesystem code. The happy path is easy. The risk lives in ordering, symlinks, and metadata leaking into the hash. I turned the task into an experiment. MonkeyCode's free model access and free server option meant the model ran on a remote server while I kept verification on my laptop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The plan: let the model write the first version, then prove or disprove it against a reference oracle. The Contract The goal was not "a tool that compiles." The goal was a tool that matches a reference implementation on every input I could generate. I wrote the contract in three sentences: Same tree → same hash, on any machine. Different content, name, or symlink target → different hash. File metadata (mtime, inode) must not affect the hash. Implementation Step 1: the prompt. I gave the model the contract, the C++17 standard, and one constraint: a single file with no dependencies beyond the standard library. Step 2: the draft. The model returned one .cpp file in a single response. It compiled on the first try. That is the exact moment where most workflows stop. This one did not. Step 3: the reference oracle. Instead of reviewing the code line by line, I built a harness that compares the tool against a shell pipeline: find " $tree " -printf '%P\0' | sort -z | wh