Gemma 4 on an Old 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6

This article provides a step by step deployment guide for Gemma 4 E2B's quantization-aware-trained (QAT) checkpoint to a local, laptop hosted GPU enabled system — a much older Lenovo Yoga 9 with a 4 GB GTX 1650 Ti. A suite of Python MCP tools is built to simplify management of the llama.cpp hosted deployment. https://github.com/xbill9/gemma4-dev/tree/main/local-llamacpp-1650ti-2b-q4_0 Gemma 4 E2B in bfloat16 is 9.5 GiB of weights. This laptop's GPU has 4 GiB. Even a plain int8 conversion would not fit. The QAT release closes that gap: the same model, trained knowing it would be stored at 4 bits, in a 3.35 GB file of which only 1.31 GiB ever has to be on the GPU. It loaded in 1618 MiB, left more than half the card free, and decodes at 73.75 tok/s. What is this project trying to Do? Every other rig in this repository serves Gemma 4 from rented hardware: Cloud TPU, Compute Engine, EC2, Cloud Run. I wanted to know whether the same model would run on the laptop I already had — a 10th-generation Core i7 with a GTX 16-series GPU, no tensor cores and a 4 GB ceiling that no quota request can raise. It does, and QAT is the reason. The rest of this article is how, and what the old hardware changes about running it. At This Point You Should Have… An NVIDIA GPU with a CUDA driver — here a GTX 1650 Ti with Max-Q Design, driver 615.71.09 The CUDA toolkit — nvcc 13.3 here Python 3.10 or newer for the MCP server — 3.14.7 here, the system python3 , no virtualenv The repository cloned, and local-llamacpp-1650ti-2b-q4_0/ as your working directory Claude Code, or any MCP client that speaks stdio The Laptop Machine Lenovo Yoga 9 15IMH5 CPU Intel Core i7-10750H, 10th generation (Comet Lake), 12 threads RAM 15 GiB, as free reports it GPU GeForce GTX 1650 Ti with Max-Q Design, 4096 MiB GPU power 40 W limit The MCP server reports the GPU the same way: gpu_status 📡 **GPU** — `local-llamacpp-1650ti-2b-q4_0` NVIDIA GeForce GTX 1650 Ti with Max-Q Design, 7.5, 4096 MiB, 1606 MiB, 2127 MiB, 615.71