Inside vLLM: Following One Request from the API to GPU Execution

Article 1 of 3 · vLLM Internals This English edition is adapted from the published Chinese article on Zhihu . It preserves the source-code references, experimental boundaries, and reproducible artifacts while adapting the structure for an international engineering audience. Series: Part 1 · Request lifecycle · Part 2 · CUDA kernels and paged attention · Part 3 · FlashAttention from PyTorch to Triton This article follows one offline inference request through vLLM V1: from LLM.generate() and inter-process communication to scheduling, input flattening, GPU model execution, paged KV-cache access, sampling, and resource reclamation. The goal is to answer one concrete question: what happens behind the call to llm.generate() before the completed result reaches the caller? The discussion assumes familiarity with Transformer inference, including prefill, decode, KV caching, and autoregressive generation. It focuses on how those concepts appear in vLLM source code rather than reteaching the model architecture. Version scope. The source references were verified against vLLM 0.22.0; this edition was checked on September 3, 2026. vLLM evolves quickly, so some filenames and call boundaries will move. The long-lived ideas—continuous batching, token budgets, paged KV allocation, and the separation between scheduling and execution—are the real subject of the article. Why Read the Source Instead of Another API Guide? Many introductions stop at the useful analogy that PagedAttention manages the KV cache much like virtual memory manages pages. The analogy does not tell us how a request is admitted, how variable-length requests become a flat token batch, or what the page table looks like at the kernel boundary. The answers are in the source. This article follows vLLM 0.22's V1 execution path from the public entry point to the CUDA boundary. It is a source-code walkthrough, not an API tutorial. Start with a System Map Start with the process boundary and the engine loop. In vLLM V1, Engin