What You Should Know About Tokens, Context, and AI Cost

Most of us use AI coding tools in a very normal way. We paste an error, ask for a fix, paste a file, ask again, run a command, paste the output, and keep going. After some time, we get a message saying something like you are out of tokens or you have reached your message limit . Most of the time, the reason is tokens. What is a token? A token is a small piece of text the model reads or writes. It can be a word, part of a word, a symbol, or spacing depending on the language and context. The model does not see text exactly like we do. It breaks everything into tokens first. So when you send a message, you are sending input tokens. When the model replies, it creates output tokens. If your coding agent reads files, terminal logs, docs, diffs, and old chat history, that can also become input tokens. What is a context window? The context window is the amount of text the model can keep in view at one time. It includes your message, the previous conversation, files, tool output, system instructions, project rules, and the model's own reply. Some models can hold a lot now. 200K tokens is already common in many coding workflows. Some newer models can go near 1M tokens. That sounds huge, and it is huge. But it does not mean you should always use it. Roughly speaking, 1M tokens can be hundreds of pages of text. It can be a big part of a codebase, many docs, or long chat history. But the model still has to read through that text. More context can mean more cost, more waiting, and more chances for the important thing to get buried. A rough mental model: Context size What it might hold 32K tokens A few files, a long bug report, or a small feature discussion 128K tokens Many files, long logs, or a decent chunk of project docs 200K tokens A large debugging session with files, logs, and history 1M tokens Hundreds of pages, big docs, or a large slice of a codebase This is not exact. Different languages, code, spacing, and tokenizers change the count. But it gives you the idea. Large c