Tracking token usage across OpenAI, Anthropic, and Gemini: every streaming gotcha I hit

OpenAI, Anthropic, and Gemini each report token usage differently, and it stops being trivia the moment you track LLM cost. I build Spanlens, an open-source LLM observability tool that sits in front of all three as a proxy and records every call with its model, latency, tokens, and cost. To do the cost part I read the token usage back out of every response, including the streaming ones. I assumed the three providers would report usage in roughly the same way. They send the same kind of data, after all: input tokens, output tokens, maybe a cached count. How different could it be. Pretty different, it turns out. Here is the whole thing in one table, then each gotcha in detail with the real parser code from the repo. Provider Where usage lives (streaming) Cache accounting Field names OpenAI final chunk, needs stream_options: { include_usage: true } prompt_tokens includes cache prompt_tokens / completion_tokens Anthropic split across message_start + message_delta input_tokens excludes cache, so add it input_tokens / output_tokens Gemini usageMetadata , two stream formats not applicable promptTokenCount / candidatesTokenCount Gotcha 1: the usage numbers live in different places in the stream For a non-streaming call this is boring. Every provider hands you a usage object on the response body and you read it. Streaming is where it gets weird, because the token counts are not in the content chunks. They show up somewhere else, and "somewhere else" is different for each provider. OpenAI puts the usage in a final chunk, after all the content, right before [DONE] . You only get it if you ask for it with stream_options: { include_usage: true } . Miss that flag and you stream the whole response and end up with no usage at all. export function parseOpenAIStreamChunk ( line : string ): Partial < ParsedUsage > | null { if ( ! line . startsWith ( ' data: ' )) return null const data = line . slice ( 6 ). trim () if ( data === ' [DONE] ' ) return null const json = JSON . parse ( data