Five ways your LLM cost tracking is lying to you

Your monthly OpenAI or Anthropic invoice tells you how much you spent. It doesn't tell you which feature spent it, which model, or why last Tuesday cost three times as much as Monday. So at some point you (or your team) will build a metering layer: wrap the client, read usage off the response, multiply by a price table, ship it to a database. I did exactly that over the past few months while building an LLM observability service, and my numbers were wrong in five different ways before they were right. Every one of these failures was silent. No exception, no alert, just numbers that were quietly too low or too high. This is the list I wish someone had handed me. Pitfall 1: Streaming responses quietly report zero tokens OpenAI's Chat Completions API returns no usage data at all for streaming requests unless you pass stream_options: { include_usage: true } . No error, no warning. The stream just never contains token counts. If your metering reads usage off the chunks, every streaming call gets recorded as 0 tokens, $0. And since chat UIs are almost always streaming, that's most of your traffic. This one bit me twice in the same audit. First finding: all streaming calls in my own dashboard were $0. Second, nastier finding: I had a budget-gate feature that blocks calls once spend crosses a limit, and it waved every streaming call straight through — because as far as it could tell, streaming was free. The fix is to inject the option in your wrapper when the caller didn't set it: let injected = false ; if ( params . stream && params . stream_options ?. include_usage === undefined ) { params = { ... params , stream_options : { ... params . stream_options , include_usage : true }, }; injected = true ; } But there's a trap inside the fix. With include_usage on, OpenAI appends one extra chunk at the end of the stream that carries usage and has an empty choices array . Any downstream code that does chunk.choices[0].delta — which is most example code on the internet — will throw