标签：#Terraform

Line 267 of a 427-line Terraform plan: # aws_rds_cluster.reporting must be replaced - /+ resource "aws_rds_cluster" "reporting" { ~ arn = "arn:aws:rds:us-east-1:842910557412:cluster:reporting" - > ( known after apply ) ~ cluster_resource_id = "cluster-D85642F9611A" - > ( known after apply ) ~ engine_version = "14.9" - > "15.4" ~ id = "reporting" - > ( known after apply ) ~ storage_encrypted = false - > true # forces replacement # (29 unchanged attributes hidden) } The merge request says "bump reporting Postgres to 15.4." The plan does exactly that. It also destroys the reporting database and creates an empty one in its place. Underneath the known-after-apply churn, two attributes are changing. One is the version bump, the thing your MR is about. The other is storage_encrypted flipping from false to true , and it isn't yours. Someone on another team that shares this repo merged it earlier in the week. You're just the one deploying. You review other people's Terraform MRs and have a feel for what each stack normally does; most weeks someone else shepherds the deploy. Today it's you. Your change goes out next, so you're carrying everything merged since the last deploy, including work you never reviewed and had no reason to know about. Nobody was negligent. The queue simply had someone else's change in it. It's a good change, by the way. You want encrypted storage. But there's no in-place path from unencrypted to encrypted on an RDS cluster. Terraform's only move is destroy and create. That's what -/+ means, and the comment at the end of the line says it in plain English: forces replacement . And the version bump alone would have failed. Going from 14 to 15 is a major version upgrade, and Aurora refuses those unless the config sets allow_major_version_upgrade = true . This one doesn't. That MR by itself would have died at apply, loudly, with an error naming the exact problem. A replacement doesn't upgrade anything. It creates a new cluster at 15.4 from scratch, so the f

If you've provisioned a VPC from a Terraform module without fully internalising what each piece is doing, that's fine — right up until something breaks. An instance that should be reachable isn't. A private instance can't pull a package update. And you're left checking five different resources with no clear mental model of how they connect. This post builds that mental model from the ground up. Not just definitions — the why behind each piece, so troubleshooting becomes deduction instead of guesswork. CIDR math you actually need A CIDR block is IP address / prefix length . The prefix length fixes the network portion; the remaining bits are your host space. Formula: 2^(32 - prefix) = total addresses . AWS reserves 5 per subnet (network address, VPC router, DNS, reserved, broadcast). CIDR Total addresses Usable /16 65,536 65,531 /20 4,096 4,091 /24 256 251 /28 16 11 To reverse-engineer a prefix from a required host count: round up to the next power of two, subtract the exponent from 32. Need 300 hosts? Next power of two is 512 (2⁹), so prefix = 32 - 9 = /23 . Run this before sizing any subnet that will host an autoscaling group or EKS node group. Start with /16 for the VPC itself. VPC CIDR is difficult to resize after the fact — once you have subnets, peering connections, or Transit Gateway attachments built against it, renumbering becomes a migration project. /16 costs nothing up front and avoids that corner. Subnet allocation: carving up the VPC A practical three-AZ production layout from 10.0.0.0/16 : Tier AZ-a AZ-b AZ-c Size Typical use Public 10.0.0.0/24 10.0.1.0/24 10.0.2.0/24 /24 ALB, NAT gateway, bastion Private/app 10.0.16.0/20 10.0.32.0/20 10.0.48.0/20 /20 EKS nodes, ECS, EC2 Data 10.0.64.0/24 10.0.65.0/24 10.0.66.0/24 /24 RDS, ElastiCache Reserved 10.0.128.0/17 /17 Future tiers, Transit Gateway, VPN The jump from /24 in the public tier to /20 in the app tier is intentional. ALBs and NAT gateways consume very few IPs; the app tier is where consumption scales

I got an AWS question and implemented it to make sure that the option is correct. A critical financial application runs on RDS for PostgreSQL. The requirements are tight: 1-second RPO, 60-second RTO, and database credentials rotated every 30 days without taking the application offline. Two independent problems. Two independent solutions. Prerequisites Check these before running terraform apply : RDS Proxy availability RDS Proxy is not available on all instance types. It requires instances with at least 2 vCPUs. db.t3.micro is not supported. db.t3.medium and above work. Terraform executor permissions The IAM principal running Terraform needs, at minimum: rds:CreateDBInstance rds:CreateDBProxy rds:CreateDBProxyTargetGroup rds:RegisterDBProxyTargets rds:ModifyDBInstance iam:CreateRole iam:AttachRolePolicy iam:PutRolePolicy iam:PassRole secretsmanager:CreateSecret secretsmanager:PutSecretValue secretsmanager:RotateSecret lambda:CreateFunction lambda:AddPermission ec2:CreateSecurityGroup ec2:AuthorizeSecurityGroupIngress ec2:CreateDBSubnetGroup AdministratorAccess on the account covers all of these. Lock it down after the initial setup. VPC requirements RDS Proxy runs inside your VPC. You need at least two private subnets in different Availability Zones. The rotation Lambda also runs inside the VPC so it can reach the RDS instance directly during the credential update step. The problem Database failure recovery RPO of 1 second means almost no data loss is acceptable. RTO of 60 seconds means the application must resume within a minute of a failure. A standard single-instance RDS setup fails both requirements: there is no automatic failover, and restoring from a backup takes far longer than 60 seconds. Credential rotation Rotating credentials on a schedule sounds simple until you factor in application downtime. If you update a password and the application still holds connections authenticated with the old one, those connections fail. The rotation mechanism needs to handle

When I first connected my GitLab CI/CD pipelines to AWS, I used the simplest solution: an IAM user with an Access Key and Secret Access Key stored as GitLab CI/CD variables. It worked. But there was one problem: those credentials were permanent. They had to be stored, protected and eventually rotated. If they were accidentally exposed in logs or compromised, they could remain valid until manually revoked. I wanted a cleaner solution. So I replaced permanent AWS credentials with OIDC federation between GitLab and AWS . The result is simple: GitLab pipelines can access AWS without storing any permanent AWS credentials. In this post, I'll explain how I implemented it, how the authentication flow works, and one important issue I faced when using it with EKS and Terraform. The architecture The authentication flow looks like this: ┌──────────────┐ │ GitLab CI │ └──────┬───────┘ │ │ OIDC token ▼ ┌──────────────┐ │ AWS STS │ └──────┬───────┘ │ │ Temporary credentials ▼ ┌──────────────┐ │ IAM Role │ └──────┬───────┘ │ ├──────────► Terraform │ ├──────────► ECR │ └──────────► EKS Instead of GitLab storing an AWS Access Key, it proves its identity to AWS using a short-lived OIDC token. AWS verifies the token and returns temporary credentials. How does OIDC authentication work? The process can be summarized in five steps: GitLab creates an OIDC token for the CI/CD job. The pipeline sends this token to AWS. AWS verifies that the token really comes from GitLab. AWS checks that the project is allowed to assume the requested IAM role. AWS STS returns temporary credentials. These credentials expire automatically. So there is nothing permanent to store or rotate inside GitLab. Step 1 — Register GitLab as an OIDC provider AWS first needs to trust GitLab as an identity provider. I configured the OIDC provider using Terraform: data "tls_certificate" "gitlab" { url = "${var.gitlab_url}/.well-known/openid-configuration" } resource "aws_iam_openid_connect_provider" "gitlab" { url = var . gi

You know the ritual. terragrunt run --all -- plan Then you scroll. Past forty units of Refreshing state… . Past the ninth identical count instance. Past a tags_all.LastModified that changes on every single run because your CI stamps a timestamp into it. Somewhere in there are the two lines you actually needed to see — probably the # forces replacement on a database. You scroll back up. You lose it. You pipe it to a file and grep for must be replaced . You approve anyway, because it's 6pm. I got tired of that, so I wrote tgsieve . What it does It runs the plan for you, reads the structured output instead of the prose, throws away the noise you declared as noise, collapses everything that repeats, and prints what's left. DESTROY / REPLACE (1) envs/prod/a ± aws_db_instance.main engine_version "14.7" → "15.3" forces replacement UPDATE (5) 5 units envs/dev/a, envs/dev/b, envs/prod/a, +2 more ~ null_resource.pin triggers.region "eu-central-1" → "us-west-2" SUMMARY ±1 replace ~5 update severity: 1 high, 5 medium hid 214 attributes across 3 rules (--explain to see them) That's five units of a real terragrunt plan — the same run terraform prints as several hundred lines. The report nests three deep — where , then what , then which fields : UPDATE (5) envs/prod/c ← the unit, said once ~ aws_s3_bucket.this ← the resource tags_all.entity "tgb" → "tgc" ← the attributes that changed A change that's identical across units replaces the directory with the set it covers, so the first column always answers the same question: where . It doesn't scrape text This matters, because the obvious implementation is fragile garbage. You might reach for terragrunt run --all -- plan -json . It doesn't work: terragrunt forwards terraform's own NDJSON straight through, so lines from units running in parallel interleave with no way to tell them apart. So tgsieve asks terragrunt for machine-readable artifacts and reads those: What Flag it passes What it gets per-unit plans --json-out-dir one tfplan.j

Pinterest has revealed the Resource Provisioner Pipeline (RPP), its own Terraform execution engine. It ensures least-privilege access and needs dual-control reviews. This is important for the company’s AWS infrastructure, as it adds strict guardrails to the GitHub Actions workflows. By Claudio Masolo

One morning, a new engineer on the team asked me a simple question. "The Lambda for the new notification feature — does it go under modules/ , or somewhere else?" I didn't have a good answer. We had a modules/lambda/ directory, so the obvious move was to put it there, and I nearly said so before something stopped me. The notification feature was part of the order workflow. Was this a reusable part, or a piece of the order domain? Two different questions were hiding inside one "where does it go?", and our directory structure couldn't tell them apart. The conversation ended the way these conversations always end. "Let's just put it in modules/lambda/ for now." The layout everyone uses You've probably seen this structure. Most Terraform repositories look like it: ├── modules/ │ ├── vpc/ │ ├── ecs/ │ ├── rds/ │ ├── iam/ │ └── lambda/ └── environments/ ├── dev/ └── prod/ It works. It plans, it applies, it looks organized. Nothing about it is wrong until the business asks for something. "Ship the new feature." "Traffic doubled, scale it up." "Compliance changed, revisit the permissions." Each request is one business change. And each one sends you into vpc/ , ecs/ , rds/ , iam/ , secrets/ , cloudwatch/ . Different requests, same sprawl. One reason to change, six directories to touch. Back when I worked this way, review time didn't go where you'd expect. Whether the change was correct was the easy part. The hard question was whether it was safe to apply, and nobody could answer that from the diff, so we asked whoever remembered what else depended on the security group being edited. Software design has a word for this: low cohesion. Things that change together are stored apart. We'd never accept this in application code. We learned — from decades of work on cohesion, coupling, and separation of concerns — to keep things that change together in one place. Somehow that vocabulary never made it down to our infrastructure repositories. This is not a Terraform problem. It is a de

My Terraform drift pipeline could detect a manual EC2 tag change, classify it as LOW, and run Terraform to remove it. Then the pipeline moved on. The evidence existed, but it was spread across CodeBuild output, Lambda logs, and an SNS message. If I wanted to know what changed, how it was classified, and whether remediation started, I had to reconstruct the event from multiple AWS services. The pipeline could act on drift. It could not remember drift. Phase 4 added that memory: a durable DynamoDB record, a read only API, and a small dashboard that turns the event history into something I can inspect without opening three AWS consoles. The Stack Terraform drift event ↓ SNS ↓ Severity Lambda ├── classifies HIGH / MEDIUM / LOW ├── starts remediation for eligible LOW drift └── writes the audit event to DynamoDB ↓ API Gateway HTTP API ↓ Read only Lambda ↓ DynamoDB Query ↓ CloudFront → static dashboard ↑ private S3 bucket The browser receives static HTML, CSS, and JavaScript from CloudFront. JavaScript calls API Gateway, the API Lambda queries DynamoDB, and the returned JSON becomes the live dashboard. There is no EC2 web server and no application process running continuously. Step 1: Store Every Classified Event I created a DynamoDB table with a composite key: resource "aws_dynamodb_table" "drift_events" { name = "terraform-drift-events" billing_mode = "PAY_PER_REQUEST" hash_key = "project" range_key = "timestamp" attribute { name = "project" type = "S" } attribute { name = "timestamp" type = "S" } } project groups the history for one Terraform project. The ISO 8601 timestamp orders its events. DynamoDB only requires attribute definitions for keys and indexes. Fields such as high_count , changes , and status still belong in each item, but they do not belong in the table schema block. I passed the table name into the existing severity Lambda instead of putting it directly in the code: environment { variables = { DRIFT_EVENTS_TABLE = aws_dynamodb_table . drift_events . name

Before you start: This picks up where Part 1 left off. From part 1, you would've learned how to setup a Bedrock Knowledge Base in the console. In addition to that, you should have a general understanding of how the ingestion and query pipeline works. Introduction & Motivation I started this project with a singular goal: to build a comprehensive Terraform module that allows developers to deploy the entire infrastructure for a "Chat with PDF" application faster. When Amazon Bedrock was first unveiled in April 2023 , I jumped in immediately. Like many of you, I built several proof-of-concepts (PoCs) through the AWS Console. The UI is amazing for building quick pocs, but once I moved into experimentation, I realized it would be best to quickly setup and tear down the infra. An example use case was testing if there were any cost savings in using S3 Vectors vs OpenSearch and how much cost savings exactly. None of the Terraform modules I found on GitHub ( at the time ) seemed to cover the end-to-end pipeline I was looking for, so I decided to build mine. I'm also big on learning so why not. What Are We Building? A couple of terraform modules to automate everything we clicked through manually in Part 1. One terraform apply brings up the full stack: S3 Bucket : your document store. Encrypted at rest, versioning on, zero public access. OpenSearch Serverless : the vector database. Stores the embeddings Bedrock generates during ingestion. Bedrock Knowledge Base : orchestrates the chunking, embedding, and storage of documents, and retrieval at query time. Ingestion Lambda : triggered automatically when you upload a file to S3. Starts a Bedrock ingestion job so documents are chunked, embedded, and indexed without ClickOps. Query Lambda : accepts a natural language question, calls RetrieveAndGenerate , and returns an answer with source citations. Full source code + ReadMe: Bedrock Project . If you run into issues or want to extend the module, feel free to open an issue. Architectu

HashiCorp has introduced tfpolicy, a new HCL-based policy-as-code framework for Terraform, now available in public beta within HCP Terraform. It is designed to simplify and modernize infrastructure governance by integrating policy creation and enforcement directly into Terraform workflows, eliminating the need for separate tools and languages. By Sergio De Simone

1. Introdução: Conectando IaC e Automação Nos artigos anteriores desta série, exploramos a poderosa combinação de Terraform e YAML para gerenciar configurações de infraestrutura em múltiplos ambientes, desde os conceitos básicos até padrões avançados de deep merge e modularização. No entanto, a verdadeira força da Infraestrutura como Código (IaC) se manifesta quando integrada a um pipeline de Integração Contínua e Entrega Contínua (CI/CD). É no CI/CD que a promessa de provisionamento automatizado, consistente e seguro da infraestrutura se torna realidade. Este artigo se aprofundará na implementação prática desses conceitos em um projeto real de CI/CD. Abordaremos a estrutura ideal do repositório, as etapas essenciais de um pipeline, estratégias de branching, considerações de segurança e as melhores práticas para garantir que sua infraestrutura seja implantada de forma eficiente e confiável. 2. Estrutura do Repositório para CI/CD Eficaz Uma estrutura de repositório bem definida é crucial para a organização e automação em um ambiente de CI/CD. Ela deve refletir a separação entre código Terraform e dados YAML, além de acomodar múltiplos ambientes e serviços. . (root do repositório) ├── README.md ├── .github/workflows/ # Ou .gitlab-ci/, .azure-pipelines/, etc. │ └── terraform.yml ├── terraform/ # Código Terraform genérico e módulos │ ├── main.tf │ ├── variables.tf │ ├── outputs.tf │ └── modules/ │ ├── vpc/ │ │ ├── main.tf │ │ └── variables.tf │ └── webserver/ │ ├── main.tf │ └── variables.tf └── config/ # Dados de configuração YAML por ambiente/serviço ├── global.yaml ├── environments/ │ ├── dev/ │ │ ├── base.yaml │ │ └── services/ │ │ ├── webapp.yaml │ │ └── database.yaml │ ├── staging/ │ │ ├── base.yaml │ │ └── services/ │ │ ├── webapp.yaml │ │ └── database.yaml │ └── prod/ │ ├── base.yaml │ └── services/ │ ├── webapp.yaml │ └── database.yaml └── services/ ├── defaults/ │ ├── webapp.yaml │ └── database.yaml └── overrides/ ├── webapp-prod.yaml └── database-dev.yaml Exp

1. Introdução: Rumo à Infraestrutura como Código de Nível Empresarial Nos artigos anteriores desta série, estabelecemos os fundamentos da separação de código e dados no Terraform com YAML (Artigo 1) e exploramos técnicas intermediárias de modularização e provisionamento dinâmico (Artigo 2). Agora, no terceiro e último artigo, mergulharemos em padrões avançados que são essenciais para gerenciar infraestruturas complexas e escaláveis em ambientes corporativos. O foco será em como lidar com hierarquias de configuração intrincadas, mesclar dados de forma inteligente e integrar essa abordagem em fluxos de trabalho de CI/CD. À medida que a infraestrutura cresce, a necessidade de abstração e automação se torna ainda mais crítica. Este artigo abordará: Deep Merge de Configurações: Como combinar dados de múltiplos arquivos YAML de forma hierárquica, onde configurações mais específicas sobrescrevem as mais genéricas. Gerenciamento de Múltiplos Arquivos YAML: Estratégias para organizar e carregar configurações de diferentes escopos (global, ambiente, serviço, região). Integração com CI/CD: Como automatizar o processo de implantação de infraestrutura usando essa abordagem em pipelines de integração contínua e entrega contínua. 2. Deep Merge de Configurações: Mesclando Dados Hierarquicamente Um dos maiores desafios ao gerenciar configurações em múltiplos níveis (global, ambiente, serviço) é a necessidade de mesclar mapas de formaprofunda, onde valores de níveis mais baixos (mais específicos) sobrescrevem ou complementam valores de níveis mais altos (mais genéricos ou padrões). A função merge nativa do Terraform realiza uma mesclagem superficial, o que significa que ela apenas mescla o primeiro nível de chaves, e se uma chave existir em ambos os mapas, o valor do segundo mapa prevalece. Para mapas aninhados, isso não é suficiente. [1] 2.1. O Desafio do merge Superficial Considere a seguinte estrutura de configuração: config/global.yaml : webserver : instance_type : t2.micro min_s

1. Introdução: Elevando a Abstração no Terraform No Artigo 1 desta série, exploramos os fundamentos da separação de código e dados no Terraform utilizando arquivos YAML e a função yamldecode . Aprendemos a carregar configurações básicas por ambiente, o que já representa um avanço significativo na organização de projetos de Infraestrutura como Código (IaC). No entanto, à medida que a infraestrutura se torna mais complexa, a simples leitura de um arquivo YAML pode não ser suficiente para manter a modularidade e evitar a duplicação de código. Este segundo artigo aprofundará nas técnicas intermediárias, focando em como combinar a flexibilidade do YAML com os poderosos recursos de modularização do Terraform. Abordaremos a passagem de configurações YAML para módulos, o uso do meta-argumento for_each para provisionamento dinâmico de recursos e módulos, e a aplicação de funções como lookup e condicionais para lidar com a variabilidade e opcionalidade dos dados de configuração. 2. Modularização com Dados YAML A modularização é um pilar fundamental para a construção de infraestruturas escaláveis e manuteníveis no Terraform. Módulos permitem encapsular um conjunto de recursos relacionados, tornando-os reutilizáveis em diferentes partes do seu projeto ou em outros projetos. Ao combinar módulos com dados YAML, podemos criar componentes de infraestrutura altamente configuráveis. 2.1. Estrutura de Projeto com Módulos Vamos expandir a estrutura de diretórios do Artigo 1 para incluir um módulo de exemplo: . (root do projeto) ├── main.tf ├── variables.tf ├── outputs.tf ├── environments/ │ ├── dev.yaml │ ├── staging.yaml │ └── prod.yaml └── modules/ └── webserver/ ├── main.tf ├── variables.tf └── outputs.tf 2.2. Definindo o Módulo webserver O módulo webserver será responsável por provisionar uma instância de servidor web (por exemplo, uma instância AWS EC2). Ele receberá suas configurações como variáveis de entrada. modules/webserver/variables.tf : variable "instance_type" { descripti

If you ship a React Native / Expo app to the App Store, you know the ritual. Open the Apple Developer portal, create a bundle identifier, tick the capability checkboxes, generate a provisioning profile, pick the right certificate. Then hop over to the Expo dashboard, create the EAS app, wire up credentials, add your environment variables one screen at a time. It works, until you have to do it again for a second app, or a second environment, or a teammate needs to know why a capability is enabled. None of it is written down. It drifts. And the usual mobile tooling doesn't help much here: fastlane and the EAS CLI are great, but they're imperative — scripts that do things — not a declarative description of what your release setup should be . That's the gap these two providers fill: elevenode/appstore — App Store Connect: bundle identifiers, provisioning profiles, certificates. elevenode/expo — Expo Application Services (EAS): apps, credentials, environment variables, update channels. Both are open source (Apache 2.0) and published on the Terraform Registry. Let's use them together to describe a mobile app's release setup as code. What you'll need Terraform (or OpenTofu) An App Store Connect API key (Users and Access → Integrations → App Store Connect API): the key, its key ID, and your issuer ID An Expo access token (expo.dev → account settings → Access Tokens) and your Expo account name Export the credentials as environment variables so nothing sensitive lands in your config: export APPSTORE_KEY = " $( cat AuthKey_XXXX.p8 ) " export APPSTORE_KEY_ID = "XXXXXXXXXX" export APPSTORE_KEY_ISSUER_ID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" export EXPO_TOKEN = "your-expo-access-token" export EXPO_ACCOUNT_NAME = "your-account-name" Wiring up both providers terraform { required_providers { appstore = { source = "elevenode/appstore" } expo = { source = "elevenode/expo" } } } # Reads APPSTORE_KEY / APPSTORE_KEY_ID / APPSTORE_KEY_ISSUER_ID from the env. provider "appstore" {} # Re

One of the biggest mistakes beginners make when learning Terraform is treating their local machine as the deployment server. A typical workflow looks like this: terraform init terraform plan terraform apply While this approach is perfectly fine for learning, it quickly becomes problematic when working on real-world projects with multiple engineers. Consider these questions: Who deployed the infrastructure? Was the infrastructure reviewed before deployment? Can someone else reproduce the deployment? What happens if the engineer's laptop is lost or misconfigured? How do we know exactly what changed? These are some of the reasons Infrastructure as Code (IaC) is almost always integrated with Continuous Integration and Continuous Deployment (CI/CD) pipelines in professional environments. In this article, we'll build a simple Terraform CI/CD pipeline using GitHub Actions. Instead of focusing only on the YAML syntax, we'll first understand why each stage exists and how they work together to produce safe, repeatable infrastructure deployments. What is Terraform CI/CD? Terraform CI/CD is the process of automating the validation, planning, and deployment of infrastructure whenever changes are made to Terraform code. Instead of running Terraform commands manually from a developer's laptop, a CI/CD platform executes those commands automatically in a controlled environment. The workflow typically looks like this: Developer │ ▼ Git Push │ ▼ GitHub Repository │ ▼ GitHub Actions │ ▼ Terraform Init │ ▼ Terraform Validate │ ▼ Terraform Plan │ ▼ Manual Approval │ ▼ Terraform Apply │ ▼ AWS Infrastructure This approach provides consistency, visibility, and security while reducing the chances of human error. Why Not Run Terraform Manually? Running Terraform from your laptop works well for personal projects, but it introduces several risks in a team environment. Manual Deployment CI/CD Deployment Requires someone to remember every command Runs automatically Easy to skip validation Validat

If you've ever cat -ed a Terraform or OpenTofu state file, you already know the uncomfortable truth: it's a plaintext JSON dump of everything your infrastructure knows, including secrets. Database passwords, generated private keys, API tokens injected through providers — they all land in state, in the clear. OpenTofu is the one place where you can fix this at the source, because native state and plan encryption is a first-class OpenTofu feature that upstream Terraform does not have. Here's how it actually works and how I roll it out on existing projects without breaking them. Why plaintext state is a real risk State is not a cache you can regenerate. It's the authoritative map between your HCL and the real resources, and OpenTofu has to store the values of attributes to compute diffs. That includes sensitive ones. Marking an output sensitive = true only hides it from the CLI output — it's still written verbatim to state. On real infra I've seen state end up in three places it shouldn't: an S3 bucket without SSE and with overly broad read IAM, a CI artifact that got uploaded to a build cache, and a developer laptop with terraform.tfstate committed to a feature branch by accident. Backend encryption (like S3 SSE) helps for one of those. It does nothing for the other two, because the moment state leaves the backend it's plaintext again. OpenTofu's encryption operates at the data layer, before the bytes ever hit the backend or a local file. The state is encrypted at rest everywhere: in the backend, in local copies, in CI artifacts. That's the property I want. Anatomy of the encryption block Encryption lives in a terraform { encryption { ... } } block. It has three moving parts: a key provider (where the encryption key comes from), a method (the actual cipher), and targets ( state and/or plan ) that bind a method to what you want encrypted. terraform { encryption { key_provider "pbkdf2" "passphrase" { passphrase = var . tofu_encryption_passphrase } method "aes_gcm" "defa

Originally published at blog.whynext.app . It started as an ordinary cleanup problem. Users upload media files (recordings and images) through presigned URLs. The server issues an upload URL, the client uploads straight to S3, then calls a commit API to say "register this key as a real asset." The problem is what happens when someone gets a presign but never commits. The app crashes, the network drops, the user leaves the screen, and the bucket is left with an object that isn't registered anywhere. I wanted a lifecycle rule to clean these up, but there was no way to write one. Committed and uncommitted objects were mixed under the same prefix, so any rule that says "delete old things" would delete real assets too. A daily upload quota kept the pile from growing fast, but the fact remained: there was no path to reclaim the space. The design: what isn't committed lives in tmp The backbone of the fix is key namespace separation. presign issues a temporary key under the tmp/ prefix. When commit passes validation (existence check via HEAD, Content-Type, size limit), it promotes the object to its final key with CopyObject and deletes the tmp original. Objects whose commit never arrives stay in tmp/ , and a lifecycle rule expires them after 7 days. Now the lifecycle rule only has to look at tmp/ . Real assets are outside its blast radius from the start. The clients didn't need to change. I read all three upload flows to confirm this: every one of them uses the key returned in the commit response for its follow-up calls, so the server can change the key shape without them noticing. Commits for old-format keys already in flight at deploy time still go through the existing path. One trap here. This bucket has versioning enabled. On a versioned bucket, expiration doesn't delete an object. It only adds a delete marker, and the original bytes stay behind as a noncurrent version. Without a paired noncurrent_version_expiration (1 day), the cleanup runs and not a single byte is rec

AWS has recently introduced CloudFormation express mode, a deployment option that can reduce infrastructure deployment times by marking stack operations complete once resource configuration is applied, rather than waiting for full resource stabilization. By Renato Losio

From Zero to a Working EKS Pipeline: Terraform, Ansible, and GitLab CI/CD (and Everything That Broke Along the Way) I recently built an end-to-end deployment pipeline on AWS EKS using Terraform for infrastructure, Ansible for configuration, and GitLab CI/CD to tie it all together. On paper, that sentence sounds clean. In practice, it took several rounds of "why is this failing" before it actually worked. This post is not a "here's how EKS works" tutorial. There are plenty of those. This is the version with the failures left in, the quota limits, the IAM permission walls, the pods that wouldn't schedule, and the resources that refused to die. If you're building something similar, I'm hoping this saves you a few hours of confused Googling. Repo: gitlab.com/nenyeonyema/terraform-eks-ansible-cicd What I Was Building The goal was a full IaC-driven pipeline: Terraform to provision the EKS cluster and supporting AWS infrastructure (VPC, node groups, IAM roles) Ansible to handle configuration tasks on top of the provisioned infrastructure GitLab CI/CD to automate the whole thing — plan, apply, configure, deploy — on every push Simple enough in theory. Four separate blockers said otherwise. Blocker #1: Free Tier ASG Restrictions The first wall I hit was with the Auto Scaling Group for my EKS node group. AWS Free Tier limits how much compute you can provision, and my initial node group sizing quietly ran into those limits — the kind of failure that doesn't always throw an obvious, single-line error. Fix: I resized the node group to stay within Free Tier boundaries and got explicit about instance types and desired/min/max capacity in Terraform, instead of leaving Auto Scaling to make assumptions I couldn't afford. Lesson: If you're building on Free Tier, hardcode your capacity expectations early. Don't let the defaults surprise you later. Blocker #2: EKS Private Endpoint Access By default, EKS clusters can be configured with private-only API server endpoint access. That's grea

In version 1.11, HashiCorp introduced Terraform Ephemeral resources and write-only attributes to allow for root configs that do not store secrets in the Terraform statefile. But many users ask about how they can adopt ephemerals. This blog attempts to lay out the ways secrets can be stored in state and how you should update your configurations to remove those secrets. Note: For a primer on ephemerals ( see this blog post ). Scenarios to consider: Data sources that fetch a static secret Resources that receive a secret Resources that generate a dynamic a secret Resources that fetch generated secrets to store in another 3rd party system Scenario 1: Data sources with static secrets Ephemeral resources can often be a drop-in replacement for data sources pulling static values: data "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } ephemeral "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } However, using these values has 1 specific difference. The attributes on a ephemeral resource are considered ephemeral and can only be used as ephemeral arguments. That means 2 places: Provider blocks Provider blocks are considered ephemeral, so ephemeral resources may populate arguments: provider "example" { password = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } Write-only arguments Write-only arguments are special arguments that require the ephemeral taint for values: resource "aws_db_instance" "example" { ... password_wo = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } If the resource you wish to pass a value to does not have an available ephemeral, open an issue with that provider. You can reference: this blog post this agent skill Scenario 2: Resources that receive a static secret Without duplicating to the section above, write-only arguments are a way to get secrets out of state. Above has guidance if the secret value comes from a data source, but what if its from a variable?