Running a Private LLM Game Master Entirely in the Browser

I recently discovered that you can run a fully interactive, narrative-driven RPG in your browser without uploading a single byte of user data to a cloud server. For a developer who is tired of the "send prompt to API, wait for response, render text" latency loop, this felt like a breakthrough. The result is Starwright , an endless space adventure where the plot is generated dynamically by a private on-device AI model. The Wedge: Latency and Privacy as Features Most browser-based AI games rely on a constant handshake with a remote inference engine. This introduces two friction points: network latency, which breaks immersion during dialogue, and privacy concerns, where your creative inputs are processed by third-party servers. By shifting the compute burden to the client using WebGPU, we can run a small model that runs in your browser entirely offline. This isn't just about cost savings on inference tokens; it’s about the feel of the interaction. When there is no network round-trip, the "typing" feel of the AI game master disappears. The narrative flow becomes immediate, similar to a traditional text adventure but with the generative flexibility of large language models. For developers building AI-native applications, this architecture suggests a shift in how we think about "always-on" AI. Instead of treating AI as a service, we treat it as a local capability. Implementation: WebGPU and Quantization The technical challenge in bringing this experience to the browser was fitting a capable narrative model into the memory constraints of a client device while maintaining responsive performance. We utilized WebGPU to accelerate the matrix multiplications required for inference, allowing the model to run smoothly on both modern desktops and capable laptops. The model is quantized to reduce its footprint, ensuring it can load within seconds. Here is a simplified view of how the inference loop is structured in the application: // Simplified inference loop for the on-device mod