The Model's JSON Was Almost Valid. I Made It Grade Its Own Homework for 48 Hours.

Every extraction pipeline I have ever pointed at a language model shares the same dirty secret: the JSON comes back almost valid. Almost is where the bugs live, because almost passes your eyes and then fails your schema at midnight. So I built a loop where the model grades its own homework, then let it run for 48 hours on a free server to see what breaks. The experiment The idea was simple: take plain-text payloads that look like webhook bodies, extract five fields against a small schema, and give the model exactly one chance to fix its own mistakes. I wrote the rules down before writing any code, because rules written after a failure are just excuses. Pass one asks the model to return the fields as JSON. A validator checks the result against the schema. If validation fails, pass two sends the original payload, the bad JSON, and the exact validation errors back to the model. Every attempt, raw text included, lands in a JSONL log. I ran that loop for 48 hours on MonkeyCode's free server option, using its free model access for both passes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Here is the loop, trimmed to the parts that mattered. import hashlib import json import time from datetime import datetime , timezone import jsonschema import requests SCHEMA = { " type " : " object " , " required " : [ " event " , " customer_id " , " amount " , " currency " ], " properties " : { " event " : { " type " : " string " , " enum " : [ " charge.succeeded " , " charge.failed " ]}, " customer_id " : { " type " : " string " , " pattern " : " ^cus_ " }, " amount " : { " type " : " integer " , " minimum " : 0 }, " currency " : { " type " : " string " , " minLength " : 3 , " maxLength " : 3 }, }, } SEEN : set [ str ] = set () def now_iso () -> str : return datetime . now ( timezone . utc ). isoformat () def call_model ( prompt : str ) -> str : # Point this at the free model endpoint you are testing. resp = requests . post ( " https://your-endpoint.e