标签：#cost

₹85,000 per month. That was the AI API bill sitting in my client's inbox when they called me in a mild panic last quarter. They run a mid-sized e-commerce operation in Pune — about 4,000 orders a day — and had integrated AI into customer support, product descriptions, and internal reporting. The AI was working beautifully. The invoice was not. Three weeks later, their monthly bill was ₹12,400. Same tasks. Same quality. No corners cut. Here's exactly what changed. The real problem: every task was using the most expensive model When I audited their setup, the issue was obvious within five minutes. Every single API call — whether it was classifying a customer complaint into one of 8 categories or generating a 2,000-word product description — was hitting the same premium model. It's the most common mistake I see with businesses adopting AI: they pick one model during the proof-of-concept phase and never revisit that decision as they scale. You wouldn't hire a senior chartered accountant to do data entry. But that's essentially what was happening — a top-tier reasoning model answering "Is this complaint about shipping or billing?" Fix 1: Model routing — the single biggest cost lever Model routing means sending each task to the cheapest model that can handle it at acceptable quality. I categorised their ~47 distinct API call types into three tiers. 68% of calls moved to the lightweight tier, 20% to mid-tier, only 12% stayed on premium. That single change dropped the bill from ₹85K to roughly ₹38K — no quality loss, verified with two weeks of A/B testing on customer satisfaction scores before switching fully. Fix 2: Prompt caching — stop paying for the same context twice Their support bot sent the same 1,200-token system prompt with every call — policies, tone, catalogue context, all identical across thousands of daily calls. Caching processes it once and references it cheaply on subsequent calls within the window. At ~6,000 support interactions a day, this alone saved ₹8,

Cloudflare announced Wallets, giving agents a stablecoin balance and spending controls, though only handle claiming is live and squatting complaints have already surfaced. Payments run on x402, now hosted by the Linux Foundation. The controls bound single payments, not sequences, leaving composition to the application above. By Steef-Jan Wiggers

Every AI video pipeline eventually has to answer an unglamorous question: what did we actually pay for that clip? On the main video-generation service, the answer for months had been "a hardcoded constant." That's fine until the vendor changes its own pricing, or a code path pays for the same synthesis twice, or a voice engine mints a clone, bills for it, and never sends it downstream. Over a ten-PR run I audited and rebuilt the voice and lip-sync pipeline from the billing layer up, then used the vendor's own SKU tiers to cut cost 7x without touching output quality. A cost model built from hardcoded constants isn't a cost model. It's a guess that happens to compile. Billing what the vendor actually charges PR #224 was workstream one of three from a sibling-tool audit: port the cost-accounting fixes that Presenter Generation and Variant Multiplier had already found, verifying each one against this repo's own code rather than assuming the same defect existed in the same place. Anthropic returns exact token counts on every response. Nothing in the pipeline read them — every charge was a hardcoded per-call constant, so the ledger and the vendor invoice diverged the moment usage drifted from whatever number had been typed in at launch. The same PR closed a second gap: two editor-facing routes could spend money — kicking off a generation, retrying a step — outside any run . A run is the unit everything else (budgets, audit trail, the cost ledger) is keyed to. A spend with no run attached is a spend the ledger can't even see, which is worse than a wrong number. Paying twice for a take the model returns unchanged PR #225 found the sibling bug's twin: some vendor calls return the exact same asset on a retry — no new synthesis happened — and the pipeline billed a second time anyway because "call succeeded" and "call did new work" were treated as the same fact. The fix is the boring, correct kind: hash the output, and only charge when the hash changes from the take you already

How We Cut AWS Staging Costs by 87% With EventBridge Scheduler No code changes. No Lambda functions. No complex scripts. Just 4 schedulers and a realization that nobody uses staging at 3am. Here's a question every engineering team should ask themselves: "When was the last time someone actually used our staging environment at 2am?" For us? Never. Not once. Yet we were paying for it — EC2 running, ECS Fargate tasks spinning, compute burning money — every single hour of every single day, including weekends, holidays, and the 21 hours per day when nobody on our team was even awake. That's the hidden tax of staging environments. And most teams never fix it because the solution feels complicated. It isn't. This is how we cut our staging compute costs by 87.5% — using AWS EventBridge Scheduler, zero Lambda functions, and zero lines of application code. The Problem: Staging Was Running 24/7 For No Reason Our staging environment had two resources running around the clock: EC2 instance — our staging app server ECS Fargate service — our backend API container Our team actively uses staging for roughly 3 hours a day . That's it. The math was embarrassing: Running: 24 hours/day Used: 3 hours/day Wasted: 21 hours/day = 87.5% of compute going nowhere Monthly cost breakdown: EC2 + ECS Fargate (24x7): ~$19.18/month EC2 + ECS Fargate (3hr/day): ~$2.40/month Monthly saving: $16.78 Yearly saving: $201.35 Reduction: 87.5% $201/year saved on staging compute alone — with 45 minutes of setup and zero application code changes. Multiply that across dev environments, QA clusters, review apps, and load test environments. The savings compound fast. The Solution: AWS EventBridge Scheduler Most engineers reach for Lambda when they need to automate AWS tasks on a schedule. That works — but it means writing code, managing runtimes, setting up CloudWatch Logs, and maintaining a function forever. EventBridge Scheduler is the better tool here. It lets you call any AWS SDK action directly on a cron sche

OVHcloud will raise prices from September, with 2026-edition gaming servers up 87 percent and other recent servers 40 to 59 percent. Founder Octave Klaba says memory cost six times more in June than a year earlier, as RAM suppliers shifted capacity toward high-bandwidth memory for AI. AWS, buying years ahead, has repriced one reserved GPU product. By Steef-Jan Wiggers

Bruna Pereira explains how DoorDash built a content-agnostic AI moderation platform. She covers replacing costly LLM-only pipelines with a hybrid pattern: using fast internal models to filter obvious cases, LLM multi-axis scoring for nuanced decisions, and no-code workflows with backtesting. Discover how this architectural pattern cut safety incidents while scaling to millions of daily messages. By Bruna Pereira

The Nvidia Cloverleaf data center partnership announced on Friday tells you where the real constraint in AI has moved, and it is not the chip. TechCrunch reported that Nvidia has taken a minority stake in Cloverleaf , a company founded in 2024 that raised $300 million that year and sits between utility companies and data centers, arranging power and site infrastructure. Nvidia did not buy a chip designer. It bought a piece of the electricity supply chain. That reframing is worth thinking about if you build software from anywhere outside a well-supplied grid. 🔌 The bottleneck moved from silicon to substations Read the deal literally. Cloverleaf's product is not compute. It is power sourcing and site infrastructure — the interconnect agreements, the substations, the land next to a utility that can actually deliver load. Nvidia buying into that layer is an admission that shipping more GPUs does not help if nobody can plug them in. This was not a one-off either. The same week, per the reporting: Deal Announced Reported size What Nvidia bought into SB Energy (OpenAI-linked, Ohio) 17 Aug 2026 $1.5 billion Data center project with an energy parent Cloverleaf 21 Aug 2026 Several hundred million (WSJ), minority stake (Reuters) Utility-to-data-center power intermediary Terms were not disclosed by either company, so treat the dollar figures as press reporting rather than filings. Key takeaway: When the company that sells the shovels starts buying the mines, the scarce input is no longer shovels. For AI in 2026, the scarce input is grid capacity. 💰 The circular money problem you should price in Here is the part I would not skip over. Nvidia sells GPUs to data centers. Nvidia is now also investing in the companies that build and power those data centers . Some of that capital flows back as GPU orders. That is not illegal or even unusual in capital-intensive industries. Telecom vendors financed carriers for decades. But it has a specific consequence for you as a buyer of compute:

Short answer: the best way to reduce LLM cost for a product catalog is to measure cost per accepted record, then route each job by difficulty. Count prompt tokens before the call, use a small model for the easy summarize/classify/extract-JSON cases, reserve a stronger model for exceptions, and batch work that can wait. The winning option is the one that preserves the required fields at the lowest cost per accepted result, not the one with the lowest token rate. I build RAG and agent features in Python, so I want this decision to survive contact with production. A catalog description such as “blue jacket, recycled nylon, sizes S–XL” looks easy until one tenant sends HTML, another sends translated attributes, and a third puts three products in one paragraph. The useful unit is therefore a tenant-scoped record with an outcome, a token count, a retry count, and a review decision. Start with the record, not the model For this workflow, the data path is short: normalize a description, count the tokens in the planned prompt, choose a route, request a typed result, validate it, and write the result and its cost to an append-only ledger. A rejected result must remain visible. Otherwise, a cheap model can appear successful simply because invalid JSON and missing attributes disappeared from the report. The ledger needs enough identity to answer a boring but important question: which tenant paid for this retry? I use a job ID, tenant ID, source revision, model route, input token count, output token count, latency, validation status, and review status. Keep the raw description out of ordinary cost dashboards when it contains customer data; the ledger can hold a reference to the source revision instead. Prompt-token counting is a gate, not a decorative metric. If an HTML description expands the prompt beyond its budget, the worker can strip markup, split the record, send it to review, or select a different route. Those choices have different quality consequences, so record the re

El 7 de abril de 2026 escribí el primer commit de lo que iba a ser mi orquestador de agentes. Era, básicamente, una pantalla. Un servidor que gestionaba varios proyectos a la vez y desde el cual podía disparar tareas de un agente de código, con un tablero al medio que mostraba en qué etapa estaba cada cosa. Si me hubieran preguntado ese día cuál era el problema que estaba resolviendo, habría contestado sin dudar: ver y lanzar . Necesitaba un lugar desde donde disparar el trabajo y mirar cómo avanzaba. Cuatro meses después, con más de dos mil tareas cerradas por ese sistema, puedo decir que esa respuesta estaba equivocada, y que el primer indicio de por qué llegó a los tres días. Los dos primeros días fueron todos de interfaz Si miro el historial de esa primera semana, es casi cómico. El ancho del panel lateral. Los tooltips con las fechas completas al pasar el mouse. Los badges de "en progreso" sobre cada etapa. Los colores por etapa del pipeline, para que se distinguieran de un vistazo. Hay un par de commits consecutivos que me gusta especialmente como retrato de ese momento. El primero pone un emoji como ícono del botón de repetición. El segundo lo reemplaza por un carácter Unicode, porque el emoji ignoraba el color que le definía por CSS y se veía siempre igual, sin importar el estado. No lo cuento para burlarme de mí mismo. Lo cuento porque es exactamente cómo se ve un proyecto cuando todavía no sabés cuál es el problema. Estaba puliendo la superficie del sistema con mucho cuidado porque la superficie era lo único que tenía enfrente. La pregunta de fondo —qué parte de este flujo tiene que decidir un modelo y qué parte no— ni siquiera me la había hecho. El 10 de abril cambió el foco Para entonces el pipeline ya tenía forma: una cadena de pasos donde un agente elegía la próxima tarea pendiente, la implementaba y después la marcaba como terminada. Los tres pasos los hacía el modelo, porque los tres estaban escritos como instrucciones dentro de las habilidades que l

JetBrains has described how it began centralising AI usage after development-related spending increased roughly tenfold in six months. Rather than restricting engineers to a small set of approved tools, the company built a shared access and accounting layer intended to preserve tool choice while giving teams greater visibility and control over consumption. By Matt Foster

We've all done it. You spin up an RDS instance to test something on a Friday afternoon. You attach an EBS volume to a t3.micro that you kill three days later — but the volume just... stays. You allocate a static IP for a load test, and the load test ends, but the IP doesn't know that. None of this shows up as an error. Nothing crashes. There's no red banner in the console screaming at you. It just quietly sits there, accruing charges, until you open your AWS bill a month later and go "wait, why are we paying for that ?" I call these zombie resources — infrastructure that's technically alive (and billing you) but functionally dead. It's happened to me enough times on side projects and client work that I've started designing a small tool to stop it: CloudZombie . Heads up: this is early. There's no working product yet — I'm validating the idea and building in the open, starting with a waitlist. If that's not your thing, no hard feelings, but if you've felt this exact pain before, I'd love your take. The problem, more specifically Cloud waste isn't usually one big dramatic mistake. It's death by a thousand cuts: Orphaned EBS volumes — you terminate an EC2 instance, but the attached volume doesn't get deleted with it, so it just sits in an "available" state forever. Idle databases — that staging or QA RDS instance nobody remembers spinning up, quietly running at under 1.5% CPU for weeks. Abandoned static IPs and load balancers — cloud providers actually charge more for an Elastic IP that isn't attached to anything, which feels like a small act of cruelty. Individually, each one is a few dollars a month. Collectively, across a handful of side projects or a growing team's sprawl of staging environments, it adds up to real money leaking out of your runway — with zero warning. What CloudZombie is meant to do The plan is for CloudZombie to connect (read-only) to your AWS account and scan specifically for these patterns: Unattached EBS volumes sitting idle Databases with sust

A payroll and HR software team rebuilt three monoliths into over 120 smaller services over five years, with no dedicated migration budget. Every new feature was built as its own service instead of changing the old ones. The article covers the pull-based migration, the tools that made this possible, how costs were kept down, and the problems the team ran into along the way. By Prashanth Pasham

Uber's "Zero Growth Stack" focuses on scalable infrastructure that separates capacity growth from business demand, reducing hardware needs while enhancing service scaling. Central to this is garbage collection optimisation. Additionally, generative AI is integrated into development, elevating developer productivity while introducing cost management measures to maintain economic efficiency. By Olimpiu Pop

Most teams blame their model provider when the inference bill spikes. They're looking at the wrong line item. The real leak is architecture — and it's the difference between a token bill that scales with value and one that scales with chaos. Here's what we see shipping agentic systems in production. The hidden multiplier: agent loops A "2-minute task" is never one call. An agent fires 30–60 tool calls per run, and most frameworks stuff the entire conversation history into every prompt. So a job you'd estimate at ~4K tokens becomes 40 calls × 8K context = 320K tokens — billed at frontier rates. Frontier pricing per call looks cheap. Multiplied by agent-loop iterations, it quietly becomes the largest line in your cloud bill. The 80/20 of inference Not every call needs a frontier model. ~80% of agent traffic is routing, extraction, formatting, classification, summarization. Trivial. Leading efficient models — including top China models — handle these at near-parity. ~20% is genuine reasoning, open-ended generation, ambiguous planning. That's where frontier earns its price. Route the 80% to efficient models and reserve frontier for the 20%. Same output quality. A fraction of the bill. A unified gateway beats a drawer of API keys The trap most teams hit: they wire 4 providers with 4 clients, then let a naïve router "roam" between them. On failover it loses cache affinity, re-embeds context, and your 1.5x cost target drifts back toward ~1x — or worse. A single OpenAI-compatible endpoint across OpenAI + Gemini + leading China models fixes this: One client, one code path. Provider pinning holds cache locality; it only fails over on hard error, not price drift. Your application code never changes when you swap a model. In SEA, "PDPA-aligned" is the baseline, not a premium For Malaysia and SEA teams, inference isn't just a cost question — it's a compliance one. PDPA requires 72-hour breach notification and a designated DPO. In-region data residency (SG-hosted) is now the defa

xAI has a bold pitch for Grok 4.5: it codes about as well as Claude Opus 4.8, but does it with roughly a quarter of the tokens. That's not a "we're smarter" claim. It's a "we're just as good for far less money" claim, which in 2026 might matter more. Someone actually put it to the test, so let me walk through what the numbers say and why you should care. The claim and the pricing Grok 4.5 landed on July 8, 2026. xAI says it matches Opus 4.8 on coding while using about 4.2 times fewer output tokens to get there. The sticker price already favors Grok. It runs $2 per million input tokens and $6 per million output. Opus sits at $5 and $25. That's less than half the price on both sides before you even factor in the token efficiency. Stack the two together and the cost gap gets dramatic. What the benchmarks say The benchmarks mostly support the marketing, with a catch. On Terminal-Bench 2.1, which measures real command-line work, Grok 4.5 scored 83.3 percent to Opus 4.8's 78.9. But on SWE-Bench Pro, the harder test of fixing real open-source bugs, Opus still comes out ahead. So the honest read is not "Grok is better." It's "Grok is about as good, for a lot less." Different claim, and a more interesting one. The hands-on test Benchmarks are one thing, real work is another. The New Stack ran a head-to-head, giving both models the same three jobs in one real Rust project (the fd file-finder) inside Cursor, and tracked every token. I'm summarizing their results here, credit to them for actually measuring it. The three tasks were a bug fix, a multi-file refactor, and a feature build. The code both models produced was nearly interchangeable, so the story came down to tokens, time, and cost. On the small bug fix, Opus actually won. Both wrote an identical fix with all tests passing, but Opus did it faster and on fewer tokens. Grok's efficiency edge showed up on the bigger jobs. On the refactor, Grok used about 197K tokens versus Opus's 954K for the same result, roughly a fifth.

A configuration change in AWS's bill computation system showed customers estimated bills in the billions and trillions of dollars for over 24 hours. AWS's own alarms detected the anomalies but failed to halt bill generation or page engineers; customer escalations alerted the company 4.5 hours later. Budget and cost anomaly alerts were disabled platform-wide during mitigation. By Steef-Jan Wiggers

The Problem with GPT-4 Pricing When I started building my AI agent hosting service, I initially planned to use OpenAI GPT-4. Then I did the math: GPT-4: ~$30 per million tokens (input) + $60 per million (output) DeepSeek Flash: ~$0.14 per million tokens That is a 200x cost difference . But Is DeepSeek Good Enough? Short answer: for most use cases, yes. I ran both models side-by-side for customer support, content generation, and code assistance. DeepSeek Flash handled 90% of tasks just as well as GPT-4. The remaining 10% (complex reasoning, nuanced writing) barely mattered for my use case. The Cache Hit Rate Secret Here is what most people miss: DeepSeek caches repeated context. With a 90% cache hit rate, the effective cost drops to ~$0.014 per million tokens. That means 100 million tokens costs about $1.40. Let that sink in. Real Numbers from My Business 24.8 billion tokens processed Total cost: ~$20 Average: $0.008 per million tokens At this rate, I can offer 100M tokens/month for $23.99 and still have 89% margin. When to Use GPT-4 Instead Be honest with yourself: Complex multi-step reasoning? GPT-4 Creative writing with specific voice? GPT-4 Everything else? DeepSeek Flash is fine The Bottom Line Do not pay 200x more for marginal quality improvement. Use DeepSeek Flash for production workloads. Save GPT-4 for the rare cases that truly need it. I run AgentChip — managed AI agent hosting powered by DeepSeek. $23.99/month with 100M tokens included.

The "Obvious" Math That's Wrong Engineer A: "Datadog is $15K/month. Prometheus is free. We should self-host." Engineer B: "But we'd need to pay an SRE to run it. That's $150K/year." Engineer A: "Prometheus doesn't need a full SRE. It's easy." Engineer B: "Famous last words." This conversation happens at every company. Both sides have points. The real math is more complex. The Total Cost Breakdown Managed (Datadog, New Relic, Dynatrace) : Licensing: $X/month (scales with hosts, events, logs) Integration time: 1-2 weeks per service Training: 1 day per new hire Ongoing: minimal Self-hosted (Prometheus + Grafana + Loki + Alertmanager) : Infrastructure: hosting costs (~$500-$5000/month depending on scale) Initial setup: 2-4 weeks of engineering time Ongoing maintenance: 10-20% of 1 FTE Upgrade costs: quarterly, each upgrade ~1 week Storage growth: ~20% per year Expertise: junior → senior SRE hire required The honest answer: managed is cheaper for teams under 50 engineers. Self-hosted becomes cheaper around 200+ engineers if you can run it well . The Real Variables It's not just licensing cost vs. hosting cost. These factors matter more: 1. Data volume growth Managed tools charge per GB ingested or per metric. If your logs 10x, your bill 10x's. Self-hosted scales linearly with compute. You control the growth. 2. Retention requirements Managed tools often charge extra for long retention. Self-hosted you store as much as your disk allows. 3. Cardinality Prometheus dies at high cardinality. Datadog handles it but charges more. High-cardinality metrics are where self-hosted breaks. 4. Incident rate Heavy incident load means heavy query load on your monitoring tools. Self-hosted needs bigger compute for this. 5. Team expertise If your team has never run Prometheus, you'll spend 6 months in the pit learning cardinality mistakes, retention tuning, and HA setups. That's not free. The Break-Even Calculation Rough calculation for a 50-engineer startup: Managed (Datadog) : - Licensi

A three-person agency received a $14,000 AWS bill in one day after attackers extracted static access keys and burned Claude invocations on Bedrock. Combined with May's DN42 incident, where an autonomous agent provisioned $6,531 of oversized infrastructure in 24 hours, practitioners warn that cloud billing lags roughly a day behind agent-speed spend. By Steef-Jan Wiggers

"The ISS radiators are expensive and heavy. We're focused on making them cheap and light."