I let an AI make phone calls, then took the word "booked" away from it
Getting an AI to place a phone call takes an evening. The trouble starts after that. It said "your table is booked" when the restaurant had said no such thing. It talked to a voicemail greeting for two minutes, politely asking for a reservation. It accepted a price above the budget because the conversation had a nice flow. All three happened on real calls to my own phone. So I built a runtime that takes the completion decision away from the model and gives it to code . https://github.com/FORIFOR/oathra What it does npx oathra demo Two AIs start a phone call in your browser, no API key. One plays a restaurant, the other wants a table. 7 pm is full, the restaurant offers 7:30, the caller takes it, gives a name, and only when the restaurant says "you're all set" does the run become MISSION COMPLETE. The result is not a summary. Every field is anchored to something the other party said. { "field" : "time" , "value" : "19:30" , "source" : "callee" , "transcript" : "7 is fully booked, but we do have 7:30." , "span" : "7:30" , "explicit" : true , "verified" : true } Completion is a formula, not an opinion: complete = connected ∧ date.verified ∧ time.verified ∧ partySize.verified ∧ confirmed.verified ∧ constraints hold If the caller says "great, that's confirmed", it is recorded and ignored. Only the callee's "you're booked" fills confirmed . The evidence rules This part is deterministic parsers and rules. No LLM. "7 is full, but 7:30 works" → no evidence for 7:00 (negated clause) "the 13th, sorry, the 14th" → the 14th survives, with a supersede trail "probably fine" → neither agreement nor confirmation price changes after "you're booked" → the confirmation goes stale and must be re-obtained "OK, I'll look elsewhere" → not an acceptance There is a command that generates ten thousand hostile clerks to break this: oathra eval --adversarial 10000 # never-confirm / wrong-restate / negate-then-offer / silent-hangup / caller-echo-trap # False Completion: 0 / 10000 adversarial run