From A10 to M60: An Architect's Journey into Azure GPU VM Sizing for Kubernetes Inference Workloads

How an unexpected regional constraint forced us to deeply understand Azure GPU VM families, naming conventions, and workload fit. Introduction As architects, we often assume that infrastructure decisions are straightforward: "The workload is already running successfully in Region A. Let's deploy the same Kubernetes workload in Region B." That's exactly what we thought. Our workload consisted of a Visual Element Detection (VED) service hosted on Kubernetes. The application uses a PyTorch model to analyze images and detect various visual elements in an image file. The service was already running successfully on a node pool backed by Azure's NVads_A10_v5 GPU VMs. Then we hit an unexpected challenge. The target region did not offer NVads_A10_v5 instances. What looked like a simple deployment exercise became a deep dive into Azure GPU virtual machine families, GPU architectures, VM naming conventions, and workload characteristics. This article shares what I learned in the hope that it helps others who find themselves evaluating Azure GPU SKUs for AI inference workloads. I am relatively new to the world of MLOps, Model deployments, GPU Workloads etc and equally interested and excited to learn more on this front. The Workload Before discussing VM selection, let's understand the workload characteristics: Model Type : PyTorch Model Size : less than 200 MB (.pth) Image Resolution : ~2000 x 2000 Expected Throughput : 5-7 requests/sec Platform : AKS (Kubernetes) Workload Type : Inference only This is important because GPU sizing should always start from the workload and not from the VM catalog. Step 1: Understanding Azure GPU VM Families Many engineers first encounter Azure GPU machines through names like: NV12s_v3 NV6ads_A10_v5 NC4as_T4_v3 ND96isr_H100_v5 The naming can be intimidating. The first breakthrough was understanding that Azure organizes GPU VMs into three primary families: N-Series ├── NV ├── NC └── ND NV Series – Visualization and Graphics NV-series VMs are designe