Free Tokens Are Not an SLO: An Ops Cost Drill for AI Batch Queues

Free Tokens Are Not an SLO: An Ops Cost Drill for AI Batch Queues This week, two numbers trended: a harness at 100%, a model at 30%. For platform teams, a better pair is queue age and deadline slack. This article is a cost drill for the simplest AI batch path: free tokens, free server, non-negotiable deadline. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. That capacity is real. It is not an SLO. The tokens cost nothing. The queue is patient. Your deadline is not. The missing variable Token cost is easy to measure. Operations cost is easy to ignore. A free endpoint converts a per-token bill into a per-hour bill. The bill becomes your time, your retries, and your queue age. This drill keeps the ledger honest. It answers one question: what does a completed request cost when the token price is zero? Topology # worker.py (minimal, single-threaded) import queue import time import csv work = queue . Queue () for i in range ( 1000 ): work . put ({ " id " : i , " prompt_tokens " : 512 , " max_tokens " : 256 }) def call_model ( payload ): # replace with your free model endpoint return { " ok " : True , " in_tokens " : 512 , " out_tokens " : 180 } completed = 0 retries = 0 started_at = time . time () while not work . empty (): item = work . get () attempt = 0 while attempt < 4 : try : call_model ( item ) completed += 1 break except Exception : retries += 1 attempt += 1 time . sleep ( 2 ** attempt ) The worker is deliberately single-threaded. Free capacity often serializes. Serialization turns a token problem into a time problem. Declared test conditions 1,000 requests. One worker process. One free model endpoint. No client-side rate limiting. Deadline: 30 minutes. Ledger: one CSV row per request. Ledger and report # cost_ledger.py import csv import time HOURLY_OPS_COST = 50.0 # loaded engineering rate, adjust def record ( item , elapsed , retries ): with open ( " ledger.csv " , " a