I Pointed Chrome's Prompt API at a 1.25 Million Character Memoir, and It Got Interesting Fast

Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every commit. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product. A straightforward engineering question: what happens when you feed a long book to an on-device language model in Chrome and start adjusting the parameters? To explore this, I built a small experiment called Gemini Nano Book Lab : a Chrome extension sidepanel that uses Chrome’s built-in Prompt API to answer questions about Richard Wagner’s My Life , while also exposing some of the underlying mechanics. The response is only part of it. The experiment also captures: Model download behavior Retrieval cost Time to first token Context window pressure Effects of different chunking strategies Places where the API works well, and where its limits become obvious If you’re an engineer interested in systems that have rough edges—and therefore teach you something—this is a useful area to explore. What the Prompt API Is Chrome’s Prompt API is part of the browser’s built-in AI features. Instead of sending prompts to a cloud endpoint, a web app or extension can request an on-device language model session and prompt it locally. Resources: The Prompt API Session management best practices Structured output for the Prompt API Built-in model management in Chrome Debug Gemini Nano Core capabilities: Local inference Streaming results Availability check before session creation Context usage measurement Events like contextoverflow (In some environments) sampling parameters like temperature and top-k This makes it more than a simple text box—it becomes an environment for experimentation. Why a Long Book? Long inputs expose the interesting problems. Short prompts hide a lot; a paragraph‑long demo can make any model look magical. A long corpus forces concrete decisions: What chunk size works well? Should chunks overlap? How many chunks should you retrieve? What latency comes from ret