Tokens per Second Benchmarks Explained: What You're Actually Measuring

What tok/s really measures, how concurrency changes it, and why a single-user benchmark is not the whole story for local LLM performance. A Few Moments Later… How Fast Is "Fast"? Every interface in the world of local AI eventually shows you that dreaded spinner, and on the wrong setup it sits there long enough that your brain supplies the meme: "A few moments later…" That pause is a number wearing a disguise. Somewhere inside your machine, the model is grinding out tokens — fragments of words — and the only question that matters is how many of them it produces per second. Tokens per second (tok/s) is the universal speedometer of local LLMs, quoted in every benchmark and every GPU review. But it is also one of the most misleading numbers in the field, because the same model can measure 45 tok/s or 793 tok/s depending on how you test it. This guide explains what the number actually means, why it moves so dramatically, and how to read a benchmark without fooling yourself. What a Token Actually Is Before speed makes sense, the unit has to. Models do not read words; they read tokens, which are chunks of text roughly three-quarters of a character on average in English. The word "calculator" might be one token or three, depending on the tokenizer, and this is not idle trivia — it is the reason the same prompt can cost a different amount across providers, as the Token Counter Calculator shows in practice. Because tokens are the unit of both billing and speed, "tokens per second" is the single number that connects all three corners of the local AI decision: how fast the model answers (tok/s), how big the model is (parameters), and what it costs to run (hardware amortized over time). A model doing 50 tok/s reads roughly 100-150 words per second — comfortably faster than you can read. A model stuck at 5 tok/s feels like a slow internet connection in 1998. The Single-User Number Is Not the Whole Story Here is the trap: most consumer benchmarks report tok/s at one user, one requ