Strategies for running AI workloads on GKE without committed quota

You’ve built your model, your training code is containerized, and you’re ready to scale up on Google Kubernetes Engine (GKE). You go to provision your nvidia-h100-80gb node pool and... QUOTA_EXCEEDED. It’s one of the most common (and frustrating) roadblocks in modern AI development. High-end accelerators like H100s, A100s, and TPUs are in massive demand, and securing permanent, on-demand quota for them can be difficult. But a lack of on-demand quota doesn't mean you're out of options. GKE provides two powerful, cost-effective strategies for acquiring these scarce resources when you can't get standard, on-demand instances: Spot VMs and the Dynamic Workload Scheduler (DWS) . Let's break down what they are, when to use each, and how to implement them. Strategy 1: Spot VMs Spot VMs are Google Cloud's excess compute capacity sold at a massive discount, up to 90% off the price of standard on-demand VMs. They are perfect for workloads that can be interrupted. The catch is that Spot VMs have no availability guarantee. Google Cloud can "preempt" (i.e., terminate) them at any time if that capacity is needed for on-demand customers. GKE gets a 30-second warning before the node is terminated. Kubernetes uses this window to gracefully shut down your application (giving non-system pods up to 15 seconds to wrap up) before the node vanishes. When to use Spot VMs for accelerators Spot VMs are ideal for workloads that are: Fault-tolerant and stateless: Your application can handle a node vanishing and having its pods rescheduled elsewhere. Batch processing: Jobs that can be easily restarted or have checkpointing built-in. CI/CD pipelines: Running tests or builds that don't need 100% uptime. How to use Spot VMs in GKE You can easily add a Spot VM node pool to your GKE Standard cluster. The key is to use Spot VMs for your workers, not your critical system pods. Create a dedicated Spot VM node pool: When creating a node pool, simply add the --spot flag and apply a taint so standard pods