标签：#kubernetes

Cloud teams waste between 40% and 60% of their infrastructure budget on a false choice: committing to reserved capacity they won't fully use or chasing spot instance savings they can't. Introduction: The Cloud Cost Optimization Dilemma Cloud teams waste between 40% and 60% of their infrastructure budget on a false choice: committing to reserved capacity they won't fully use or chasing spot instance savings they can't operationalize. The decision between commitment discounts and spot instances is not a preference. It is a calculation with three variables: workload predictability, failure tolerance, and the operational cost of managing interruptions. Commitment discounts lock you into capacity for one or three years. You pay upfront or monthly for compute resources whether you use them or not. The mechanism is simple: cloud providers offer 30% to 72% discounts because they can forecast their own capacity planning when customers commit. You save money when your actual usage matches your commitment. You lose money when usage drops below the committed level because you still pay for idle capacity. Spot instances offer 70% to 90% discounts by selling unused cloud capacity at auction prices. The provider can reclaim these instances with 30 seconds to 2 minutes of notice. You save money when your workload can tolerate interruptions and you build automation to handle instance termination. You lose money when interruptions cause failed jobs that must restart from scratch, consuming more compute time than the discount saved. Most engineering teams pick one strategy and apply it everywhere. This creates two failure modes. Teams that over-commit pay for capacity during low-traffic periods. Teams that over-rely on spot instances spend engineering time rebuilding checkpoint systems and retry logic that costs more than the discount delivers. The correct approach is workload-specific. Measure your actual usage patterns for 30 days. Calculate the cost of interruption handling. Then a

Terraform can provision an AWS EC2 instance and set up a Python virtual environment in a single, reproducible run — the whole workflow is declarative and version‑controlled. 📑 Table of Contents 💻 Terraform — How to Provision an EC2 Instance 🔧 AWS Provider — Configuring Credentials 🐍 Python Environment — Setting up a Virtualenv on the Instance 📦 Installing Python and venv 📦 Activating and Using the Environment 📦 User Data — Automating Installation with Terraform 🟩 Final Thoughts ❓ Frequently Asked Questions How do I store the Terraform state securely? Can I use a different Linux distribution for the EC2 instance? Is it possible to attach an Elastic IP to the instance? 📚 References & Further Reading 💻 Terraform — How to Provision an EC2 Instance A Terraform configuration file describes the desired state of AWS resources; applying it makes the real cloud match that state. First, install Terraform (version 1.5.0 or newer). The binary is a single executable, so the operating system loads it directly into memory and the process performs HTTP requests to AWS endpoints. $ terraform version Terraform v1.5.0 on linux_amd64 + provider registry.terraform.io/hashicorp/aws v5.12.0 Next, create a main.tf that declares an aws_instance resource. The provider block authenticates with AWS using either environment variables or a shared credentials file. # main.tf terraform { required_version = ">= 1.5.0" required_providers { aws = { source = "hashicorp/aws" version = "~> 5.12" } } } provider "aws" { region = "us-east-1" } resource "aws_instance" "app_server" { ami = "ami-0c02fb55956c7d316" # Amazon Linux 2 instance_type = "t3.micro" # User data will be defined later user_data = data.template_file.init.rendered tags = { Name = "terraform-ec2-python" } } Running terraform init contacts the provider registry, downloads the provider plugin, and stores it under .terraform . The generated .terraform.lock.hcl file records exact plugin checksums, guaranteeing that subsequent runs use the same

A pattern keeps repeating itself in AI projects. The model works. The demo works. The proof of...

In the summer of 2016, Pokémon GO launched to a user base roughly fifty times larger than its capacity planning had anticipated. The engineering team had done load testing. They had throughput thresholds. They had autoscaling configured. Within hours of launch, the service was degraded globally — not because the infrastructure could not scale, but because it scaled too slowly against an arrival rate that exceeded every modelled scenario, and because the metric that was driving scaling decisions (CPU utilisation) lagged behind the actual saturation signal by several minutes. By the time CPU registered critical, the request queue had already grown to the point where p99 latency had crossed into the range where users were abandoning sessions faster than new sessions were being created. The engineering post-mortem identified the same root cause that appears in the post-mortems of most capacity-related incidents: the organisation's operational metrics were measuring how hard the infrastructure was working, not how much work the service could safely accept. CPU percentage is a resource utilisation metric. Memory percentage is a resource utilisation metric. IOPS is a resource utilisation metric. None of them is a service throughput metric. None of them tells you, with precision, at what arrival rate your SLO begins to degrade. Safe Operating Throughput is that metric. It is not a new concept in queueing theory or systems engineering — the idea of a safe operating ceiling predates modern distributed systems. What is new is its treatment as a first-class SRE metric: formally derived from load test data and SLO targets, continuously monitored for drift, and operationally enforced as a constraint in autoscaling configuration, capacity planning decisions, and deployment pipeline gates. Why Existing Capacity Metrics Are Insufficient The canonical capacity management approach in most organisations works like this: observe CPU or memory utilisation, set an autoscaling threshold (t

Three weeks ago, one of the teams we work with had a checkout outage. The root cause a malformed...

Kubernetes networking is one of the most misunderstood parts of running containerized workloads. A pod can reach another pod by IP — but why does that stop working after a deployment? A service exists and resolves in DNS — but traffic isn't arriving at the application. An Ingress resource is configured — but requests return 502. These puzzles are common and they stem from the same root: Kubernetes networking has several distinct layers, each solving a different problem, and it's easy to conflate them. This article walks through how Kubernetes networking actually works at each layer — from pod networking to services to Ingress to network policy — so the next time something breaks, you have a mental model to reason from. The fundamental promise: flat pod networking Kubernetes makes one core promise about networking: every pod can communicate directly with every other pod in the cluster without NAT. Every pod gets a real IP address from the cluster's pod CIDR range, and those IPs are routable between pods regardless of which node they're running on. This is not something Kubernetes itself implements. It's a contract that every Kubernetes-conformant CNI (Container Network Interface) plugin must fulfill. When you install Calico, Cilium, Flannel, Weave, or any other CNI, you're installing the component that actually creates this flat network. The mechanism varies — Flannel uses VXLAN overlays, Calico can use BGP for direct routing, Cilium uses eBPF — but the result is the same: pod-to-pod communication without NAT. Here's what a pod's network namespace looks like: $ kubectl exec -it my-pod -- ip addr 1: lo: ... 3: eth0@if12: ... inet 10.244.1.15/24 brd 10.244.1.255 scope global eth0 $ kubectl exec -it my-pod -- ip route default via 10.244.1.1 dev eth0 10.244.0.0/16 via 10.244.1.1 dev eth0 The pod has an IP ( 10.244.1.15 ) on a /24 subnet. The node this pod runs on has an IP from the same range — or a different /24 within the same /16. Traffic from this pod to 10.244.2.8 (

📌 This article was originally published on Sherdil E-Learning . I'm republishing it here so the dev.to community can benefit too. The Kubernetes vs Docker question is one of the most common sources of confusion for developers entering DevOps. People hear both names constantly, see them used together in job listings, and assume they must be competitors. They are not. Docker and Kubernetes do different jobs, and most modern infrastructure uses both. This guide explains what each tool actually does, how they fit together in a real deployment, the practical difference between Docker Compose and Kubernetes, and which one you should learn first. Docker: the container creator Docker is a tool for building, running, and managing containers . A container is a lightweight, portable package that contains an application together with its dependencies, runtime, system libraries, environment variables, and configuration files. The same container runs the same way on a laptop, a CI runner, a production server, or a cloud platform. In a typical Docker workflow you: Write a Dockerfile that describes how to build the image Run docker build to produce the image Run docker run to launch a container from it For multiple containers (a web app plus a database, for example), you use Docker Compose to define the whole set in a docker-compose.yml file and start them with one command. Docker is excellent for individual containers and small multi-container applications. The limitation is scale. What happens when you need a hundred containers across a dozen servers? When one container crashes at 3 a.m.? When you need to roll out a new version without downtime? Docker alone does not solve those problems. For the official reference, see docs.docker.com . Kubernetes: the orchestration layer above Docker Kubernetes (often shortened to K8s ) is an open-source platform that runs containers across many machines as a single coordinated system . It was originally built at Google, based on their internal

Introduction One of the common tasks performed by DevOps Engineers is upgrade of their organization's Kubernetes Cluster at least once every 3 months as Kubernetes release newer version while maintaining on the last 3 released versions. For instance, if the newest version is v1.34, the supported versions would be v1.34, v1.33 & v1.32. Hence, the need to understand how this upgrade process can be achieved with zero downtime. Prerequisites: Cordon your Nodes: This simply means making your nodes unschedulable. No new deployments would be scheduled on the node. Review and understand the change logs in the release notes - Ensure that the change logs or updated components won't affect your production environment. Kubernetes upgrade are irreversible - You can't downgrade your cluster after an upgrade. A fresh installation would be required in the event of an issue with the upgraded version. Hence Lower Level Environment Test (Unit, Staging or Pre-Production) - Given that Kubernetes upgrades are irreversible, always test the newer version and allow monitoring for about 2-weeks before production cluster upgrade. Control Plane & Nodes should be on the same versions. Cluster Auto-Scaler: If you are using this feature within your Kubernetes environment, ensure that it is on the same or compatible version with your control plane to avoid issues during the cluster upgrade. IP Addresses: Make available at least 5 IP addresses within the cluster subnet. Kubelet: This component should also match the version of your control plane before the upgrade. What are the actual upgrade processes Control Plane Upgrade: If using the Managed Kubernetes Cluster (EKS, AKS, GKS), the Cloud Company will take care of managing the control plane. However, upgrade of the cluster doesn't happen automatically. Hence, you will be required to action this via the CLI, UI or EKSCLI etc. Node Group or Data Plane Upgrade: Managed Node Groups - This is easier because you can use the rollout deployment approach,

After migrating Spark pipelines to Azure Kubernetes Service, two infrastructure settings interacted destructively: spark.kubernetes.local.dirs.tmpfs=true backed shuffle spill with RAM instead of disk, and a hard podAffinity rule forced all executors onto one node. Together, they caused repeated OOM kills invisible to standard diagnostics. By Pranav Bhasker

Learn strategies for building interrupt-resilient AI workloads on Google Kubernetes Engine (GKE).

Introduction Managed Kubernetes platforms such as Amazon EKS, Google Kubernetes Engine (GKE), and Azure Kubernetes Service (AKS) abstract away much of the operational complexity involved in running Kubernetes clusters. While this significantly improves developer productivity, it also hides many of the internal systems responsible for cluster orchestration, networking, node registration, and workload scheduling. As a result, many engineers interact with Kubernetes daily without fully understanding the components that keep a cluster operational behind the scenes. To better understand Kubernetes from an operational perspective, I set out to build and operate a self-managed Kubernetes platform on AWS using kubeadm. Unlike lightweight local environments such as Minikube or kind, kubeadm bootstraps Kubernetes in a way that closely resembles how real-world self-managed clusters are provisioned and operated. The objective of this project was not simply to install Kubernetes, but to explore: How the control plane components interact. How worker nodes register with the cluster. How Kubernetes networking behaves. How cloud integrations work. How traffic reaches workloads running inside the cluster. How operational failures surface during deployment and runtime. How production-style systems behave beneath managed abstractions. This article documents the architecture, implementation process, engineering decisions, operational lessons, and troubleshooting insights encountered during the effort to bring the platform to a healthy operational state. Project Objectives The primary objectives of this project were to: Provision infrastructure on AWS using Terraform. Bootstrap a self-managed Kubernetes cluster using kubeadm. Configure Kubernetes networking using Calico. Integrate Gateway API with AWS Load Balancer Controller. Expose workloads externally using AWS Application Load Balancers. Validate cluster functionality through application deployment. Understand the operational mechani

You’ve built your model, your training code is containerized, and you’re ready to scale up on Google Kubernetes Engine (GKE). You go to provision your nvidia-h100-80gb node pool and... QUOTA_EXCEEDED. It’s one of the most common (and frustrating) roadblocks in modern AI development. High-end accelerators like H100s, A100s, and TPUs are in massive demand, and securing permanent, on-demand quota for them can be difficult. But a lack of on-demand quota doesn't mean you're out of options. GKE provides two powerful, cost-effective strategies for acquiring these scarce resources when you can't get standard, on-demand instances: Spot VMs and the Dynamic Workload Scheduler (DWS) . Let's break down what they are, when to use each, and how to implement them. Strategy 1: Spot VMs Spot VMs are Google Cloud's excess compute capacity sold at a massive discount, up to 90% off the price of standard on-demand VMs. They are perfect for workloads that can be interrupted. The catch is that Spot VMs have no availability guarantee. Google Cloud can "preempt" (i.e., terminate) them at any time if that capacity is needed for on-demand customers. GKE gets a 30-second warning before the node is terminated. Kubernetes uses this window to gracefully shut down your application (giving non-system pods up to 15 seconds to wrap up) before the node vanishes. When to use Spot VMs for accelerators Spot VMs are ideal for workloads that are: Fault-tolerant and stateless: Your application can handle a node vanishing and having its pods rescheduled elsewhere. Batch processing: Jobs that can be easily restarted or have checkpointing built-in. CI/CD pipelines: Running tests or builds that don't need 100% uptime. How to use Spot VMs in GKE You can easily add a Spot VM node pool to your GKE Standard cluster. The key is to use Spot VMs for your workers, not your critical system pods. Create a dedicated Spot VM node pool: When creating a node pool, simply add the --spot flag and apply a taint so standard pods

There's a question that keeps surfacing across engineering teams — sometimes in architecture reviews,...

Appendix: Live System Output — Real Pipeline in Production All output below was captured live from the running pipeline on 2026-03-08. These are not mock outputs — they come from actual AWS infrastructure and Kubernetes clusters. ArgoCD — All 50 Applications Across 6 Clusters The following is the live output of argocd app list from the hub cluster ( myapp-production-use1 ). Every component of the pipeline is represented — security, logging, monitoring, backups, and the application itself. $ argocd app list --output wide NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH argocd/argo-rollouts-myapp-production-use1 myapp-production-use1 argo-rollouts production Synced Healthy argocd/argo-rollouts-myapp-production-usw2 myapp-production-usw2 argo-rollouts production Synced Healthy argocd/aws-lbc-myapp-production-use1 myapp-production-use1 kube-system production Synced Healthy argocd/eso-myapp-production-use1 myapp-production-use1 external-secrets production OutOfSync Healthy ← known false positive argocd/eso-myapp-production-usw2 myapp-production-usw2 external-secrets production OutOfSync Healthy ← known false positive argocd/falco-myapp-dev-use1 myapp-dev-use1 falco production Synced Healthy argocd/falco-myapp-dev-usw2 myapp-dev-usw2 falco production Synced Healthy argocd/falco-myapp-production-use1 myapp-production-use1 falco production Synced Healthy argocd/falco-myapp-production-usw2 myapp-production-usw2 falco production Synced Healthy argocd/falco-myapp-staging-use1 myapp-staging-use1 falco production Synced Healthy argocd/falco-myapp-staging-usw2 myapp-staging-usw2 falco production Synced Healthy argocd/fluent-bit-myapp-dev-use1 myapp-dev-use1 logging production Synced Healthy argocd/fluent-bit-myapp-dev-usw2 myapp-dev-usw2 logging production Synced Healthy argocd/fluent-bit-myapp-production-use1 myapp-production-use1 logging production Synced Healthy argocd/fluent-bit-myapp-production-usw2 myapp-production-usw2 logging production Synced Healthy argocd/fluent-bit-myapp-

DevSecOps Pipeline — Completion Runbook All code is written and pushed to GitHub. This runbook covers the remaining operational steps: Terraform applies, GitOps ARN updates, and ArgoCD deployment. Prerequisites Install these tools if not already present: # AWS CLI v2 winget install Amazon.AWSCLI # Terraform 1.6+ winget install HashiCorp.Terraform # Terragrunt # Download from https://github.com/gruntwork-io/terragrunt/releases # Place in C:\Windows\System32\ or add to PATH # kubectl winget install Kubernetes.kubectl # ArgoCD CLI winget install argoproj.argocd AWS Profile Setup The root terragrunt.hcl uses profiles named myapp-{env}-{region_alias} . Configure them in ~/.aws/config : [profile myapp-production-use1] region = us-east-1 role_arn = arn:aws:iam::591120834781:role/AdministratorAccess source_profile = default [profile myapp-production-usw2] region = us-west-2 role_arn = arn:aws:iam::591120834781:role/AdministratorAccess source_profile = default [profile myapp-staging-use1] region = us-east-1 role_arn = arn:aws:iam::690687753178:role/AdministratorAccess source_profile = default [profile myapp-staging-usw2] region = us-west-2 role_arn = arn:aws:iam::690687753178:role/AdministratorAccess source_profile = default [profile myapp-dev-use1] region = us-east-1 role_arn = arn:aws:iam::557702566877:role/AdministratorAccess source_profile = default [profile myapp-dev-usw2] region = us-west-2 role_arn = arn:aws:iam::557702566877:role/AdministratorAccess source_profile = default PHASE 1 — Terraform Applies Work from the myapp-infra/ directory. Run in the order shown — capture outputs for updating GitOps files in Phase 2. 1.1 WAF (production + staging) # Production us-east-1 terragrunt apply --terragrunt-working-dir live/production/us-east-1/waf # Output → webacl_arn (copy this value) # Production us-west-2 terragrunt apply --terragrunt-working-dir live/production/us-west-2/waf # Output → webacl_arn (copy this value) # Staging (no GitOps ARN needed, but good to have) terra

Threat-intel reports from recent years document campaigns in which attackers obtain AWS IAM credentials from developer workstations, use them to enumerate cloud accounts and access Kubernetes clusters. From there, attackers deploy poisoned container images to move laterally and harvest secrets. The MITRE ATT&CK chain maps to: T1552.001 ( Credentials in Files ) → T1078.004 ( Valid Accounts: Cloud Accounts ) → T1610 ( Deploy Container ) → T1496 ( Resource Hijacking ). This is not an isolated case. The Shai-Hulud supply chain attack harvested Kubernetes credentials from CI and developer workstations, feeding exactly this kind of attack chain. This research started with a short list of questions: What are Kubernetes secrets, exactly? What can an attacker do with them? How can defenders harden their clusters? So before we look at what we found in the wild, and how to harden clusters to mitigate impacts, let's define what Kubernetes secrets are. Three Surfaces, Three Secret Formats A simplified view of the cluster has three sides that matter for this post: A developer, or automation pipeline, talks to the Kubernetes API server with credentials. That is the canonical front door. On every node, the kubelet exposes its own HTTPS API. The same credentials can authenticate to it directly when it is reachable on the network. The cluster's nodes pull images from container registries (Docker Hub, GitHub Container Registry, ECR, Quay, GitLab, ACR…) using a second set of credentials. Kubernetes Architecture These are the attack surfaces where leaked secrets have the most impact, and three secret formats unlock them: TLS client certificates are used by humans through a kubeconfig file with the kubectl command to connect to a Kubernetes cluster. JSON Web Tokens, or Service Accounts, are non-human identities (NHI) used to automate cluster operations from CI/CD jobs, controllers, and integrations. By default, JWTs have no expiration date — which is why a JWT leaked years ago can still

Platform Engineering Labs has announced a major update to its open-source Infrastructure-as-Code platform, formae, introducing full Kubernetes support, native Helm integration, direct .tfvars compatibility, and a new public plugin hub aimed at simplifying cloud-native infrastructure management By Craig Risi