"How Does LLM Actually Work? From Prompt to Prediction"

Large Language Models have quickly become part of everyday software development. We ask them to explain code, debug errors, generate tests, write Python scripts, summarize documentation, or help us understand an unfamiliar codebase. Within seconds, we get a response that can feel surprisingly natural. But what actually happens during those few seconds? Suppose you type: What is a build system? The model doesn't simply search through a database for a stored answer, and it doesn't generate the entire response in one shot. At the heart of an autoregressive LLM is a deceptively simple task: Given the tokens I've seen so far, what token should come next? Getting to that prediction, however, involves several layers of computation. At a high level: Prompt ↓ Tokens ↓ Embeddings ↓ Transformer ↓ Logits ↓ Next Token ↓ Repeat Let's follow that journey. 1. Everything Starts With the Prompt Consider: What is a build system? Humans immediately recognize the words and their meaning. A neural network needs numbers. Before the model can process the question, the text passes through a tokenizer . 2. Tokenization: Breaking Text Into Pieces A tokenizer divides text into smaller units called tokens . Conceptually, our prompt might become: ["What", " is", " a", " build", " system", "?"] This is only an illustration. Actual tokenization depends on the tokenizer used by the model. A token isn't necessarily a complete word. It might represent: a complete word part of a word punctuation whitespace combined with text a number part of an identifier a programming-language symbol Each token is mapped to an integer called a token ID . Conceptually: ["What", " is", " a", " build", " system", "?"] ↓ [3923, 374, 264, 1975, 1887, 30] The IDs above are illustrative. The important part is the transformation: Human-readable text has become a sequence of numbers the model can process. But token IDs themselves don't capture useful semantic relationships. The number 1975 , for example, doesn't inherently ex