Your Bill Doubled Overnight: A Triage Runbook

An LLM bill that doubles overnight has one of about eight causes, and the fastest route to it is not reading code. It is six queries over your request log, run in order, each of which eliminates a branch. The first one takes thirty seconds and settles whether you are looking for more requests or dearer ones. Before the queries: stop the bleeding If spend is still climbing while you investigate, put a ceiling on it first. A provider-side spending limit, a lowered rate limit on your own gateway, or disabling the newest feature flag all buy you time, and none of them require knowing the cause. Diagnosis is cheaper when the meter is not running. Resist the urge to change several things at once to make it stop. If you disable three suspects simultaneously and the spend falls, you have solved the incident and learned nothing, and it will return. What you need logged The runbook assumes one row per request. If you do not have this, building it is the first fix, and it is a day of work that pays for itself the first time this happens. CREATE TABLE llm_requests ( ts timestamptz NOT NULL , request_id text , model text NOT NULL , -- from the RESPONSE, the resolved one route text , -- which feature or endpoint caller text , -- service, job, or user id tenant text , -- customer, if multi-tenant prompt_tokens int NOT NULL , cached_tokens int , -- prompt tokens served from cache completion_tokens int NOT NULL , reasoning_tokens int , cost_usd numeric ( 12 , 6 ), -- computed at write time status int , attempt int , -- 1 for the first try, 2+ for retries duration_ms int ); Two columns do disproportionate work. attempt is what makes a retry storm visible instead of looking like organic traffic. And model taken from the response rather than the request is what makes an alias move visible — the request said one thing and the provider served another. The six queries, in order Volume or unit cost? Everything downstream depends on this answer, and it is one query. SELECT date_trunc('day',