Standing Up a GPU Cluster on AKS for vLLM

This article is Part of a series on running vLLM on AKS and walks through creating an AKS cluster with a GPU node pool, deploying vLLM onto it, and wiring up Prometheus and Grafana for visibility. Companion pieces: Choosing the right GPU | Why your autoscaler flaps | Source Setup Summary Cloud: Azure GPU node: Standard_NV36ads_A10_v5 (1× A10, 24 GB) Image / model: vllm/vllm-openai:latest serving Qwen/Qwen2.5-7B-Instruct-AWQ Observability: kube-prometheus-stack (Prometheus + Grafana), KEDA, NVIDIA DCGM exporter All commands below are bash. The steps are ordered and each one depends on the previous. Dependency chain The build order follows one chain: model → VRAM requirement → GPU SKU → region availability → quota. Step 0 — Prerequisites (one-time, survives resource group deletion) GPU quota. Request through Portal → Quotas → Compute → This article: Requested Standard NVADSA10v5 Family vCPUs = 108 in westus (108 = 3 nodes × 36 vCPUs, matching the autoscaler's max-count 3 set in step 3). Quota is granted per-subscription and survives resource group deletion, so this step happens once, not on every rebuild. A quota is Azure's per-subscription limit on how much of a resource (here, GPU vCPUs in a specific VM family) you're allowed to provision at once. New subscriptions start at 0 for GPU families since it's expensive and can be abused. You need it because without an approval, az aks nodepool add for a GPU will fail outright. The request goes through manual Azure approval, so it has to happen before you plan to build. * Prerequisites * Existing Azure Subscription: Local tooling: Helm 3+ kubectl Bash Bash Variables to set for use through the setup RG = <resource-group-name> CLUSTER = <cluster-name> LOCATION = <preferred-location> Step 1 — Create Resource group az group create -n $RG -l $LOCATION Step 2 — Create AKS cluster, on a CPU system pool az aks create -g $RG -n $CLUSTER \ --node-count 1 --node-vm-size Standard_D2s_v5 \ --generate-ssh-keys The GPU does not go on thi