How We Translate 300-Page Books Using Claude Without Hitting Token Limits

Breaking long documents into overlapping chunks, preserving context, and reassembling with FastAPI At LectuLibre, we’ve built an AI‑powered platform that translates entire books—EPUBs and PDFs—using large language models. When we first hooked up Claude’s API, we naively fed it a 300‑page PDF in one request. It failed immediately. Claude 3 Opus has a 200K token window, but a 300‑page book can easily run to 300K tokens or more. Even if we squeezed it in, the output would be truncated and the quality would degrade at the extremes of the context window. So we faced a classic long‑document problem: how do you translate a book that’s larger than the model’s context window? Here’s the real approach we ended up with, the code we wrote, and the lessons we learned. The Problem: Token Limits Are Real Claude 3 Opus and Haiku models (and most LLMs) have a maximum context length—200,000 tokens for Opus. A token is roughly ¾ of a word. A 300‑page novel with ~75,000 words translates to about 100K tokens, so it should fit, right? But translations from English to Spanish can expand by 15–20%, and the prompt instructions, system message, and the user message itself all eat into that budget. Plus, we needed to send the entire source text in every call to give the model full context. That’s not feasible. We could have tried a simple split: cut the book at arbitrary page boundaries and translate piecemeal. That fails spectacularly. Narrative breaks mid‑sentence, and phrases like “the previous chapter” lose their referents. We needed a more intelligent chunking strategy. Our Approach: Sliding Window with Overlapping Paragraphs We settled on a sliding window chunking algorithm based on paragraphs, with a generous overlap. Here’s the idea: Split the source text into paragraphs (using \n\n ). Build chunks of max_chunk_tokens (we used 180,000 to keep a safety margin), adding paragraphs one by one and counting tokens with tiktoken . When the chunk exceeds the limit, we start a new chunk but we