Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't

Serving Gemma 4 E2B on a TPU v6e-1 A Cloud TPU v6e-1 (Trillium) costs 2.25× a v5e-1 and returns 1.62–1.68× the throughput on workloads that fit in a v5e, and 2.32–2.77× on workloads that do not. Per output token that makes v6e 34–39% dearer in the first regime and 3–19% cheaper in the second — so the case for the bigger chip is narrower than the memory ratio suggests, and break-even sits at roughly 270,000 KV tokens. v6e is not a general upgrade over v5e. It is a memory upgrade sold at a compute price: 32 GB against 16, a KV pool of 1,151,744 tokens against 321,376 (3.6×) , for 1.907× the bandwidth. Where the extra memory does nothing, the workload pays 2.25× for 1.6×. Two findings drive the rest: There is no capacity knee at any occupancy tested. TTFT = −8542 + 265 × concurrency , R² = 0.999996 , across 56% to 157% of the KV pool, with num_preemptions_total = 0 in every cell. A line fitted entirely below 100% occupancy predicts 157% to within 0.13%. Spot is more expensive than flex-start on this chip — $1.4033 against $1.35/chip-hr in us-east5, reversing the v5e ordering. The cheaper option is also the preemption-free one that stops billing by itself. Configuration: v6e-1 ( ct6e-standard-1t , one Trillium chip), vllm/vllm-tpu:nightly , vLLM 0.26.1rc1.dev256+gf5bb701fa , tpu-inference JAX backend, google/gemma-4-E2B-it at bf16, TP=1, max_model_len 32768, max_num_batched_tokens 4096, kv_cache_dtype=auto , prefix caching on. OUTPUT_LEN 128 throughout. v5e-1 comparison figures are from the same model and engine family on v5litepod-1 and are not a controlled A/B — read them as shape, not delta. Part 1 — Getting a v6e-1 at all 1.1 The gcloud spelling table On v5e, v5e is spelled v5litepod to gcloud. On v6e the marketing name and the CLI value coincide — which teaches a habit that breaks on the next chip. Context v5e single chip v6e single chip Prose, directory names v5e-1 / v5e1 v6e-1 / v6e1 --accelerator-type v5litepod-1 v6e-1 Flex-start runtime version v2-alpha-tpuv5-l