Building an Offline AI Note-Taking App with WebGPU

For the last few months, I’ve been obsessed with a specific problem: the friction between privacy and utility in modern AI tools. Most "private" AI solutions still rely on a local LLM running on your CPU or GPU via a heavy desktop application. They require installation, constant background processes, and often struggle with performance on older hardware. I wanted to see if we could do better. I wanted to see if we could run a capable language model entirely within the browser, using only the device’s hardware acceleration, with zero data leaving the machine. The result is PrivateScribe, a tool I built to handle note summarization, email drafting, and rewriting. But more importantly, it’s an experiment in what’s possible when you treat the browser not just as a display layer, but as a compute engine. The Wedge: WebGPU and True Offline The core constraint that drove this project was simple: nothing leaves the device. In the current landscape, "on-device AI" often means "installed on your device." This is fine for desktop apps, but it creates silos. You can’t easily share a workflow across a Chromebook, a Windows machine, and an iPad without installing three different native applications. By leveraging WebGPU, PrivateScribe runs entirely in the browser. This unlocks a few critical advantages: Zero Installation: Users open a URL and start working. No downloads, no permission dialogs for file system access beyond what’s needed for the session. Hardware Acceleration: WebGPU allows the browser to tap directly into the GPU. This is crucial for inference speed. A small model that runs in your browser can process text significantly faster than a CPU-bound implementation, especially on modern laptops with integrated graphics. True Offline Capability: Because the model weights are loaded locally via WebAssembly and the inference happens on-device, the app works completely offline. If you lose your internet connection in the middle of drafting an email, the AI doesn’t stop. It c