标签：#Infrastructure

The dashboard says forty instances, up from twelve this morning. The autoscaler did its job: it saw latency climb and threw hardware at it. And latency got worse. Not flat. Worse. You're paying for three times the compute to serve a slower product. Somewhere under all forty of those boxes is a single thing they're all waiting in line for, and every instance you add makes the line longer. Horizontal scaling multiplies work that doesn't have to coordinate. The instant the work does have to coordinate, more instances make it slower. Amdahl wrote this down in 1967: the serial fraction of a job sets a hard ceiling on how much faster you can go, no matter how much hardware you throw at the parallel part. Neil Gunther's Universal Scalability Law goes further: past a certain point, the cost of nodes coordinating with each other bends the curve back down. Add capacity, get less throughput. That ceiling was not set by the autoscaler, and it will not be moved by the autoscaler. It was set a long time before this morning, in a room, by whoever decided where the state lives and who has to touch it at the same instant. Now hand the service to a fleet of agents. It writes you something that looks built to scale: stateless handlers, a tidy repo, green tests, a canary that bakes fine at 1% traffic. Every gate you trust says ship it. And the bottleneck is sitting right there in the design, invisible to all of it, because the mistake isn't in the lines, it's in the shape. You cannot catch a shape problem by reading a diff. Name the hot state before you pick a framework. Where does the contended state live, and which requests touch it at the same instant? Answer that out loud, before anyone opens an editor. The tool is downstream of that answer, every time. Originally published at https://imacto.com/writing/scale-is-a-design-not-a-dial . Written with Claude Opus 4.8.

“We have no choice but to address the challenges created by these massive facilities,” New York governor Kathy Hochul said. The executive order will pause construction for one year.

I didn’t set out to diagnose hyperscalers. I wasn’t doing a grand industry analysis. I wasn’t mapping global architecture. I wasn’t trying to understand cloud strategy. I was just trying to use a popular software provider — and everything kept breaking. Every time something failed, I followed the thread. And every thread led to the same architectural gap. Eventually I realised I hadn’t been analysing hyperscalers at all. I’d accidentally mapped the substrate failure across the entire industry. Once you see the pattern, you can’t unsee it. Across Microsoft, AWS, Google, and Meta, the same structural drift appears: meaning drift identity drift trust drift state drift execution drift provenance drift agentic drift Different companies. Different stacks. Different histories. Same substrate gap. And it’s not just me. The world is waking up to these problems too. Vendor lock in isn’t just a technical nuisance anymore — it’s becoming a public conversation. People are asking why their money keeps disappearing into the same handful of providers. Organisations are asking why their systems collapse the moment they try to leave. Governments are asking why critical infrastructure depends on architectures they cannot inspect, cannot govern, and cannot reproduce. What started as a personal frustration with a popular software provider turns out to be the same structural issue everyone else is now discovering. And sovereignty is entering the conversation — not as a political slogan, but as an architectural question. When national systems depend on fragmented substrates owned by a tiny cluster of vendors, sovereignty becomes a structural issue. The question isn’t “who controls the cloud?” It’s “who controls the substrate the cloud is built on?” Follow the thread far enough and you reach a scenario nobody wants to think about: what happens in a moment of global stress when a hyperscaler’s fragmented substrate becomes a single point of failure? Not a political crisis — a structural one.

The Cybersecurity and Infrastructure Security Agency (CISA) has issued a postmortem on a data leak in which a contractor published dozens of internal CISA credentials -- including AWS Govcloud keys -- in a public GitHub repository for almost six months before being notified by KrebsOnSecurity. Experts say the gaps identified in the agency's initial response provide important lessons that all security teams should absorb.

Bryan Oliver discusses the frontier of AI infrastructure: chaos engineering for large-scale GPU clusters. He shares how engineering leaders can handle complex topologies, network protocols like RDMA, and NUMA misalignments. Discover seven practical fault-injection strategies to maximize multi-million dollar hardware efficiency and build robust observability loops. By Bryan Oliver

The company is taking a modular approach to designing these chips, anticipating that their needs will change as AI evolves rapidly by the time the chips are in production.

In version 1.11, HashiCorp introduced Terraform Ephemeral resources and write-only attributes to allow for root configs that do not store secrets in the Terraform statefile. But many users ask about how they can adopt ephemerals. This blog attempts to lay out the ways secrets can be stored in state and how you should update your configurations to remove those secrets. Note: For a primer on ephemerals ( see this blog post ). Scenarios to consider: Data sources that fetch a static secret Resources that receive a secret Resources that generate a dynamic a secret Resources that fetch generated secrets to store in another 3rd party system Scenario 1: Data sources with static secrets Ephemeral resources can often be a drop-in replacement for data sources pulling static values: data "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } ephemeral "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } However, using these values has 1 specific difference. The attributes on a ephemeral resource are considered ephemeral and can only be used as ephemeral arguments. That means 2 places: Provider blocks Provider blocks are considered ephemeral, so ephemeral resources may populate arguments: provider "example" { password = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } Write-only arguments Write-only arguments are special arguments that require the ephemeral taint for values: resource "aws_db_instance" "example" { ... password_wo = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } If the resource you wish to pass a value to does not have an available ephemeral, open an issue with that provider. You can reference: this blog post this agent skill Scenario 2: Resources that receive a static secret Without duplicating to the section above, write-only arguments are a way to get secrets out of state. Above has guidance if the secret value comes from a data source, but what if its from a variable?

AWS has outlined how ProGlove, an industrial-wearables manufacturer, was able to scale its SaaS platform to run more than one million AWS Lambda functions spread across thousands of dedicated customer accounts. By Matt Foster

Static analysis isn't just for application source code. Terraform, Pulumi, OpenTofu, and CloudFormation files are code too — and they get misconfigured just as often as a backend service. A public S3 bucket, a security group open to 0.0.0.0/0 , or an unencrypted RDS instance are all bugs you can catch before apply ever runs. TFSec is the tool most people reach for first, but it's not the only option on the OWASP Source Code Analysis Tools list . In this article I'll use Checkov , a free, open-source policy-as-code scanner built by Bridgecrew (now part of Palo Alto Networks), to scan a Terraform project end to end — from a local scan to a GitHub Actions gate that blocks merges on critical misconfigurations. The same approach works with OpenTofu and Pulumi projects too, since Checkov understands HCL directly and also has native support for Pulumi's rendered plan output and CloudFormation/ARM/Kubernetes manifests. Why Checkov? 100% open source (Apache 2.0), actively maintained, thousands of built-in policies. Understands Terraform, OpenTofu, CloudFormation, Kubernetes, Helm, Dockerfile, ARM, Serverless Framework, and Pulumi (via cdktf /synthesized plans) — one tool across most of your IaC surface. No account or API key required to run locally or in CI. Supports custom policies written in Python or YAML if the built-in rule set doesn't cover something specific to your org. ## 1. The sample infrastructure A small AWS setup with a few intentionally introduced misconfigurations — the kind that get merged during a rushed sprint: # main.tf provider "aws" { region = "us-east-1" } resource "aws_s3_bucket" "data" { bucket = "company-app-data-bucket" } # Vulnerable: bucket has no encryption, no versioning, and is publicly readable resource "aws_s3_bucket_acl" "data_acl" { bucket = aws_s3_bucket . data . id acl = "public-read" } resource "aws_security_group" "web" { name = "web-sg" description = "Allow web traffic" # Vulnerable: SSH open to the entire internet ingress { from_port

Vermilion Cliffs Ventures announced Wednesday the close of a $25 million Fund II.

Why Every Modern SaaS Needs a Billing Engine (Not Just a Payment Gateway) When building a SaaS product, API platform, AI application, or cloud service, most teams start by integrating a payment gateway like Stripe or Paystack. That solves one important problem: Collecting payments. But collecting payments and managing billing are two completely different challenges. As products grow, pricing becomes more sophisticated. Customers expect usage-based billing, flexible subscriptions, prepaid credits, seat-based pricing, tiered plans, and transparent invoices. Suddenly, the billing logic inside the application becomes one of the most complex parts of the entire system. This is why more engineering teams are separating payment processing from billing infrastructure . Payment Processing vs Billing Infrastructure A payment gateway is responsible for moving money. A billing platform is responsible for determining what should be charged, when it should be charged, and why it should be charged. Although these responsibilities often get grouped together, they solve very different problems. Payment providers typically handle: Payment authorization Card processing Refunds Payment retries Bank settlements Customer payment methods A billing engine manages: Usage metering Subscription billing Pricing rules Billing cycles Invoice generation Revenue analytics Customer usage tracking Payment orchestration Multi-gateway billing Proration Credits and quotas Keeping these responsibilities separate makes systems easier to maintain, easier to scale, and much more flexible when pricing changes. Why Billing Logic Doesn't Belong Inside Your Application Many engineering teams start with a simple subscription model. Then the business evolves. Suddenly the product needs to support: API usage billing AI token billing Storage billing Compute billing Per-seat pricing Hybrid subscriptions Enterprise contracts Prepaid credits The application slowly becomes filled with billing-specific code. Developers

Decentralized Autonomous Organizations are often presented as simple governance systems: token holders create proposals, vote, and execute decisions on-chain. In practice, building a production-grade DAO is far more difficult. A DAO is not only a smart contract. It is a distributed coordination system that combines governance logic, treasury security, token economics, identity, off-chain infrastructure, and human decision-making. A failure in any one of these layers can compromise the entire organization. Below are some of the most important technical problems DAO developers must solve. 1. Governance Attacks Through Borrowed Voting Power Many DAOs calculate voting power based on the number of governance tokens held at a specific moment. This creates a serious attack surface when tokens can be borrowed through lending protocols or flash loans. An attacker may temporarily acquire a large amount of voting power, submit or approve a malicious proposal, and return the borrowed assets shortly afterward. The standard defense is snapshot-based voting power. Instead of checking a user’s current balance, the governance contract reads historical balances from a previous block. function getVotes( address account, uint256 blockNumber ) public view returns (uint256) { return token.getPastVotes(account, blockNumber); } However, snapshots alone do not solve every problem. Developers should also consider proposal delays, minimum token-holding periods, quorum requirements, and vote-delegation risks. 2. Dangerous Proposal Execution The most sensitive part of a DAO is usually the executor. A successful proposal may call arbitrary contracts, transfer treasury assets, upgrade protocols, or change governance parameters. If proposal calldata is incorrectly validated, a governance action can execute unintended operations. A DAO should clearly separate: Proposal creation Voting Proposal queuing Timelock execution Emergency cancellation Using a timelock gives token holders and security teams

Introduction AI governance is often framed around risk, ethics, safety, and international cooperation. These are essential, but they are not sufficient. Governance only becomes real when countries have the computing infrastructure required to run, monitor, and maintain modern AI systems. Without compute, governance is theory. With compute, governance becomes capability. This article explores the missing execution layer in global AI governance — and why bridging the AI divide requires far more than policy alignment. The Hidden Dependency: Governance Assumes Infrastructure Most governance frameworks implicitly assume that nations already have: access to high performance compute reliable data pipelines secure storage operational tooling energy capacity connectivity skilled operators But this assumption is false for the majority of the world. The global AI divide is not primarily about access to models. It is about access to the infrastructure required to run them. Governance frameworks that ignore this reality risk becoming aspirational rather than actionable. The Execution Layer: Where Policy Meets Reality The execution layer is the part of AI governance that turns policy into practice. It includes: compute infrastructure data ingestion and processing pipelines monitoring and evaluation tooling human in the loop operational workflows maintenance and lifecycle management energy and cooling requirements secure deployment environments This layer is rarely discussed in governance conversations, yet it is the foundation upon which all responsible AI depends. Without an execution layer, governance collapses into paperwork. The Real Global Divide Isn’t About Models — It’s About Compute There is a persistent misconception that the AI divide is about access to large models. In reality, the divide is driven by: insufficient compute unreliable infrastructure lack of operational capacity limited data availability absence of secure environments dependency on external providers A c

In this episode, Alex Zenla (CTO/Co-founder, Edera) challenges the "laissez-faire" attitude toward modern infrastructure. She promotes "spite-driven development", building software to solve genuine technical pain points rather than passively accepting flawed abstractions, as a philosophy of improving the world of software. By Alex Zenla

In February, city officials in Cheyenne, Wyoming discovered something in their reclaimed water system that shouldn't have been there: Cupriavidus gilardii , a rare metal-resistant bacterium traced to wastewater discharges from Meta's $800 million data center campus. The contamination shut down Cheyenne's reuse water system for months , and on July 2, the city publicly named Meta's construction entity — a shell company called Goat Systems LLC — as the source. 📖 Read the full version with charts and embedded sources on ComputeLeap → "It's a very, very unpleasant surprise," said City Councilman Pete Laybourn. It shouldn't have been a surprise at all. Cheyenne is just the latest community learning what happens when AI's insatiable demand for compute meets the physical world: contaminated water, noise that residents describe as "living in hell," electricity bills that spike 267%, and — in the most surreal twist — a federal government that deleted its own energy conservation pages while a heatwave slammed the eastern seaboard. The AI industry talks endlessly about parameters, benchmarks, and scaling laws. But the story converging across Reddit, Hacker News, X, and YouTube this week isn't about models. It's about watts, gallons, and the communities living next to the machines. The water problem is worse than you think A Brookings Institution analysis puts the numbers in perspective: a typical data center consumes 300,000 gallons of water every day — equivalent to roughly 1,000 households. Large facilities gulp up to 5 million gallons daily, matching the needs of a town of 50,000. And water demand for data center cooling may rise by 870% as the current build-out continues. The scale is hard to overstate. According to a Consumer Reports investigation , Phoenix-area data centers currently use 385 million gallons annually — a figure projected to explode to 3.7 billion gallons once planned facilities come online. About two-thirds of data centers built since 2022 sit in water-st

Understanding the 550 SPF Check Failed Error The "550 SPF Check Failed" error indicates that a receiving mail server rejected an incoming email. This rejection occurs because the sender's domain failed its Sender Policy Framework (SPF) validation. SPF is an email authentication protocol defined in RFC 7208 . SPF helps prevent email spoofing. It allows domain owners to specify which mail servers are authorized to send email on behalf of their domain. Receiving mail servers perform an SPF check by querying the sender's DNS for an SPF TXT record. If the sending server's IP address is not listed in the domain's SPF record, the SPF check fails. The receiving server then rejects the email based on its configured policy, often resulting in a 550 error. This error protects recipients from unauthorized emails and enhances email security. Initial Diagnosis: Identifying the Root Cause Diagnosing an SPF failure requires examining the bounce message and the domain's DNS records. The bounce message often provides specific details about the SPF failure. Look for phrases like "SPF validation failed," "unauthorized sender," or "IP address not permitted." Common reasons for a 550 SPF Check Failed error include: Missing SPF Record: No SPF TXT record exists for the sending domain. Incorrect SPF Syntax: The SPF record contains errors, making it unreadable or invalid. Incomplete SPF Record: The SPF record does not list all legitimate sending IP addresses or hostnames. DNS Lookup Limit Exceeded: The SPF record requires more than 10 DNS lookups, violating RFC 7208. DMARC Policy Enforcement: A DMARC (Domain-based Message Authentication, Reporting, and Conformance) policy ( RFC 7489 ) with p=reject or p=quarantine is in place, enforcing strict SPF failure handling. To begin diagnosis, use our SPF checker to verify your domain's SPF record and its validity. This tool quickly identifies syntax errors and lookup issues. Step-by-Step Troubleshooting and Resolution Resolving SPF failures involves

Introduction to DMARC and the p=quarantine Policy DMARC (Domain-based Message Authentication, Reporting, and Conformance), defined in RFC 7489 , is an email authentication protocol. It builds upon SPF and DKIM to provide domain owners with the ability to protect their domain from unauthorized use. DMARC enables senders to specify how receiving mail servers should handle unauthenticated emails originating from their domain. It also provides a mechanism for receiving servers to report back to the domain owner about authentication results. DMARC policies dictate the action receiving mail servers should take when an email fails DMARC authentication. The three primary policies are: p=none : Monitor mode. Receiving servers take no action on failed messages but send reports. This is the initial deployment phase. p=quarantine : Receiving servers should treat failed messages as suspicious. They are typically placed in the recipient's spam folder or flagged for further review. p=reject : Receiving servers should outright reject messages that fail DMARC authentication. This is the strongest enforcement policy. Implementing p=quarantine is a critical step towards full domain protection. It allows domain owners to mitigate spoofing and phishing attempts without immediately blocking legitimate, but misconfigured, email streams. This policy provides a balance between security enforcement and minimizing potential deliverability disruptions. Prerequisites for DMARC p=quarantine Implementation Before deploying a p=quarantine policy, proper configuration of SPF and DKIM is mandatory. DMARC relies on these underlying authentication mechanisms and their alignment with the sending domain. SPF (Sender Policy Framework) SPF, specified in RFC 7208 , allows domain owners to publish a list of authorized sending IP addresses in their DNS. Receiving mail servers check the SPF record to verify if an incoming email originated from an authorized server. An SPF record is a TXT record at the root of

Cycle recently introduced a separate EU-based control plane, allowing European customers to keep platform management data and telemetry within Europe. The new offering is designed to improve compliance, operational isolation, and responsiveness for European organizations. By Renato Losio

A company shifted from project- to product-thinking after their platform outgrew single-team use. The limitations that they felt with their platform were one-off deliveries, lack of product vision, and weak feedback loops. They have moved toward a self-service, API-driven, multi-tenant infrastructure with clearer ownership and better abstractions. By Ben Linders

The actor and investor is joining forces with Morgan Beller, who was previously a GP at NFX, to invest in early-stage startups.