标签：#Optimization

Netflix is moving toward the open-source Apache Flink Autoscaler for more than 30,000 streaming jobs across multiple AWS regions. The operator-level approach addresses limitations of Netflix’s cluster level autoscaler for complex, stateful pipelines. Netflix reports a 58% reduction in annualized Flink compute expenditure for one team, saving approximately $1.1 million annually. By Leela Kumili

The cloud bill is the only number in most companies that nobody on the team owns until it's already a problem. Engineering owns latency. It owns error rates, p99, uptime, the whole observability wall. Finance owns the invoice. And between those two ownerships there's a gap wide enough to drive a fifth to a third of your cloud spend straight into a wall — which, across the industry, is roughly what happens. The fix isn't a smarter spreadsheet at month-end. Real cloud cost management isn't an accounting function at all — the fix is to stop treating the bill as accounting and start treating it as a product metric: cost per request, cost per tenant, cost per feature, sitting on the same dashboard as latency and error rate, owned by the same people who own those numbers. That's the whole argument. The rest of this post is why it's true and how it's done. The bill is a lagging accounting artifact, and that's the bug Here's how cloud cost is treated almost everywhere. A bill arrives. Someone in finance reconciles it against a budget. If it's higher than expected, a thread gets opened, an engineer gets pulled in, and everyone spends a week spelunking through Cost Explorer trying to reconstruct why a number that's already been spent is what it is. Then it happens again next month. Every part of that loop is broken. The signal arrives weeks after the decision that caused it. The person reading the signal can't act on it. The person who can act on it never sees it. And the unit of measurement — total dollars — tells you nothing about whether the spend was good . A bill that doubled because you doubled revenue is a triumph. A bill that doubled because someone left a debug log streaming to an expensive tier is a fire. Total dollars can't tell those two apart. They look identical on the invoice. This is the same mistake we'd never make with any other production signal. Nobody reviews latency once a month from a PDF. Nobody waits for finance to tell engineering that p99 regressed.

Most S3 bills are wrong, and the fix takes an afternoon. The data sits in the most expensive class AWS offers (S3 Standard, $0.023/GB-month), nobody set a lifecycle policy, incomplete multipart uploads are silently billing for storage you can't even see in the console, and every byte your EC2 fleet pulls from S3 is routed out through a NAT Gateway when a free VPC Gateway Endpoint would do the same job for $0. None of this needs an architecture rewrite. It needs a checklist run in the right order. Here is the order. The savings depend entirely on your access pattern — I will not promise you a number I can't see — but the mistakes below are so common that the question is usually how much , not whether . One number is just arithmetic: cold data that moves from S3 Standard ($0.023/GB-month) to Glacier Deep Archive ($0.00099/GB-month) drops about 96% on the storage line for those bytes, and on an observability platform I ran — logs aged past 90 days into Deep Archive — that is exactly the lever that did the work. S3 cost optimization is the same boring discipline as the rest of the bill: see it, then decide what each byte should actually cost. First, see the bill before you touch it You cannot optimize what you cannot measure, and S3's default billing view tells you nearly nothing useful. Turn on S3 Storage Lens before anything else. The free tier gives you 62 metrics at the bucket level with 14 days of history, and crucially it includes cost-optimization metrics out of the box — including "Incomplete multipart upload bytes greater than 7 days old," which is the single most common source of money disappearing into storage nobody knows exists ( AWS S3 Storage Lens docs , accessed 2026-06-18). Storage Lens free metrics answer the three questions that decide everything that follows: Which buckets hold the most bytes? What storage class is that data sitting in right now? Where are the incomplete multipart uploads? For deeper per-prefix analysis or a longer history, Advanced

For AWS Savings Plans vs Reserved Instances, the default answer is: buy a Savings Plan, not a Reserved Instance. The exception is OpenSearch, Redshift, and (until December 2025) databases, which still need the older Reserved model. That is the whole decision. For most teams a Compute Savings Plan is the right default: same discount as a Convertible Reserved Instance, far less to manage, and it follows your workload across instance families, regions, Fargate, and Lambda. The cases where a Reserved Instance still wins are narrow and specific, and the December 2025 launch of Database Savings Plans shrank them further. This post is the decision — RI vs Savings Plan, when each wins. It is not a deep-dive on how Savings Plans work under the hood; I cover the mechanics — how the $/hour commitment gets applied, the billing-hour math, the queue order against On-Demand — in a separate post. Here I only want to answer the question you actually have when the Cost Explorer recommendation pops up: which one do I buy? (Commitments are step three of a full bill audit — where they sit in the order is in what I'd audit first on a $50K AWS bill .) The short version Both Reserved Instances and Savings Plans are the same trade: you promise AWS a one- or three-year commitment, AWS gives you a discount over On-Demand. The difference is what you commit to . A Reserved Instance commits you to a specific instance configuration — family, and depending on type, size, region, OS, tenancy. A Savings Plan commits you to a dollar amount of usage per hour (e.g. "$10/hour of compute"), and AWS applies that discount to whatever matching usage you actually run. The Savings Plan is the more flexible instrument at the same discount level, which is why AWS itself now recommends Savings Plans over Reserved Instances for compute. ( AWS, Compute Savings Plans and Reserved Instances , accessed June 2026.) The Reserved Instance has not gone away — but for EC2 compute, it is mostly the legacy choice now. The d

If your AWS bill has a NAT Gateway line, you are paying twice for the same packet: once for the gateway to merely exist, and again for every gigabyte it carries. The fix for most teams is dull and free. Add an S3 and a DynamoDB gateway endpoint, route the heavy traffic away from NAT, and only then argue about anything fancier. That single change is free to turn on, takes minutes, and stops the most expensive traffic from ever touching the meter. This is a playbook, not a lecture. The trick with NAT Gateway pricing is that the two charges hide in different places on the bill, so most teams only ever see half of it. Numbers first, then the fixes, in the order I would actually do them. What you are actually being charged for NAT Gateway has two charges, and people forget the second one until they read the bill closely. Hourly charge — you pay for every hour the gateway is provisioned and available, whether or not a single byte moves through it. In us-east-1 (N. Virginia) and us-east-2 (Ohio) this is $0.045 per NAT Gateway-hour . That is roughly $32.85 a month per gateway just to keep the lights on. Partial hours bill as full hours. Data processing charge — you pay $0.045 per GB processed through the gateway, in the same region, on top of the hourly charge. This applies to every gigabyte, inbound or outbound, regardless of source or destination. And then there is the part the pricing page mentions almost in passing: standard AWS data transfer charges still apply on top. NAT processing is an extra meter on traffic you were already paying to move. The hourly charge is fixed and visible. The per-GB charge is the one that catches teams out, because it scales with traffic you mostly cannot see: package installs, container image pulls, S3 reads from private subnets, telemetry shipped out, cross-region calls. The rate varies by region (it runs higher in places like São Paulo, where both the hourly and per-GB rates sit around $0.093), so check your own region rather than trusti

Give me read access to a $50,000/month AWS account and I will tell you within a day where the first 20-30% is hiding, because on a mid-size bill it is almost always hiding in the same four places, in the same order: data transfer you can't see in the console, instances sized for a load test that ran two years ago, on-demand pricing on a baseline that never moves, and storage rotting in the most expensive class AWS sells. None of this needs an architecture rewrite. AWS cost optimization, at least the first and biggest pass of it, is just the bill read in the right order by someone who knows where AWS buries the meter. This is the order I work. It is the same audit I run on every account I'm handed, and it is the offer — if you want me to run it on yours, the post ends with how. But you can run most of it yourself today, and you should, because nobody is going to care about your bill as much as you do. A note before the recipe: I deal in ranges, not promises. The exact saving on your account depends on what you've built. What I can promise is that the mistakes below are common enough that the question is usually how much , not whether . Hour zero: get the real bill, not the dashboard Before touching a single resource, I want the granular data. The AWS console's cost dashboard rounds, groups, and hides the things that matter. Two tools give you the truth. Cost Explorer , with rightsizing recommendations turned on, is the fast view — group by service, then by usage type, and the bill stops being one big number and starts being a list of decisions. Resource-level and hourly granularity costs extra ($0.01 per 1,000 usage records per month), but for one audit pass it's worth pennies. The Cost and Usage Report (CUR) is the ground truth — line-item, hourly, every charge AWS makes, delivered to your own S3 bucket. Generating it is free; you pay only the few cents of S3 storage. If you're going to do this seriously, set up CUR (now delivered via AWS Data Exports) on day one. E

If you are an active user of AI-powered IDEs like Cursor, VS Code with Copilot, or Windsurf, you already know the sinking feeling of seeing this notification: "You have used 100% of your fast premium requests for this billing cycle." Suddenly, your snappy, context-aware coding assistant slows to a crawl or starts racking up expensive pay-as-you-go API bills. At the same time, you are likely paying $20/month for a ChatGPT Plus or Team subscription that sits underutilized in a browser tab. You use it for general questions, but it lacks direct, real-time access to your local codebase, forcing you to engage in a tedious dance of copying and pasting code blocks. What if you could bridge this gap? What if you could let ChatGPT Web do the heavy reasoning and planning using your local context, while saving your premium IDE tokens for fast auto-completions ? In this article, we’ll explore a highly novel, intermediate-level setup that does exactly this. By leveraging the Model Context Protocol (MCP) , Node.js , and secure Cloudflare Tunnels , you can route heavy code-planning tasks directly to your web-based ChatGPT Plus subscription safely and completely free of extra token charges. The Philosophy: Let ChatGPT Think, Let Your IDE Work When building complex software with AI, your workflow generally splits into two distinct phases: Reasoning & Planning (High Token Usage): This is where you ask the AI to read 10 source files, understand the architecture, design a new feature, or find a subtle bug. This consumes massive amounts of context window tokens. Execution & Autocomplete (Low Latency): This is where the AI writes single lines of code, refactors a function, or autocompletes your imports. This requires fast, inline API queries. Paying premium API rates (per token) for Phase 1 is incredibly expensive. This is where this open-source MCP bridge project shines. It exposes a read-only view of your local project as an MCP server. Your web-based ChatGPT (via custom GPTs or MCP int

Why shorter outputs can cost more, and how GitHub Copilot reduces wasted work across the complete coding task. The post How we make AI coding more cost efficient without sacrificing task quality appeared first on The GitHub Blog .

Microsoft expanded Foundry's model router from two regions to 28 for global standard and 21 for data zone deployments, while adding Claude Opus 4.8 and GPT-5.6 and removing four deprecated models. Default deployments receive pool changes automatically; configured subsets exclude new models until added. The effective context window equals the smallest model in the pool. By Steef-Jan Wiggers

₹85,000 per month. That was the AI API bill sitting in my client's inbox when they called me in a mild panic last quarter. They run a mid-sized e-commerce operation in Pune — about 4,000 orders a day — and had integrated AI into customer support, product descriptions, and internal reporting. The AI was working beautifully. The invoice was not. Three weeks later, their monthly bill was ₹12,400. Same tasks. Same quality. No corners cut. Here's exactly what changed. The real problem: every task was using the most expensive model When I audited their setup, the issue was obvious within five minutes. Every single API call — whether it was classifying a customer complaint into one of 8 categories or generating a 2,000-word product description — was hitting the same premium model. It's the most common mistake I see with businesses adopting AI: they pick one model during the proof-of-concept phase and never revisit that decision as they scale. You wouldn't hire a senior chartered accountant to do data entry. But that's essentially what was happening — a top-tier reasoning model answering "Is this complaint about shipping or billing?" Fix 1: Model routing — the single biggest cost lever Model routing means sending each task to the cheapest model that can handle it at acceptable quality. I categorised their ~47 distinct API call types into three tiers. 68% of calls moved to the lightweight tier, 20% to mid-tier, only 12% stayed on premium. That single change dropped the bill from ₹85K to roughly ₹38K — no quality loss, verified with two weeks of A/B testing on customer satisfaction scores before switching fully. Fix 2: Prompt caching — stop paying for the same context twice Their support bot sent the same 1,200-token system prompt with every call — policies, tone, catalogue context, all identical across thousands of daily calls. Caching processes it once and references it cheaply on subsequent calls within the window. At ~6,000 support interactions a day, this alone saved ₹8,

Cloudflare announced Wallets, giving agents a stablecoin balance and spending controls, though only handle claiming is live and squatting complaints have already surfaced. Payments run on x402, now hosted by the Linux Foundation. The controls bound single payments, not sequences, leaving composition to the application above. By Steef-Jan Wiggers

Bruna Pereira explains how DoorDash built a content-agnostic AI moderation platform. She covers replacing costly LLM-only pipelines with a hybrid pattern: using fast internal models to filter obvious cases, LLM multi-axis scoring for nuanced decisions, and no-code workflows with backtesting. Discover how this architectural pattern cut safety incidents while scaling to millions of daily messages. By Bruna Pereira

Why Is Overdraw Particularly Important in WebGL? When optimizing a Unity WebGL application, developers often focus on polygon count, draw calls, texture size, and shader complexity. But there is another performance problem that can quietly become expensive: UI overdraw . A UI may look simple to the player while the GPU is actually rendering the same pixels several times. A typical game interface might contain: A full-screen background Transparent panels Multiple decorative images Buttons Shadows Icons Text Semi-transparent overlays Popups Individually, these elements may seem harmless. When stacked together, however, they can significantly increase the number of pixels the GPU needs to process. Unity identifies overlapping transparent UI, sprites, and particles as common contributors to overdraw. So, is UI overdraw really killing your WebGL performance? Let's understand when it matters and how to reduce it. What Exactly Is UI Overdraw? Overdraw occurs when the same screen pixel is rendered multiple times during a frame. Imagine a 1920×1080 WebGL game with this UI: Full-screen background ↓ Transparent dark overlay ↓ Semi-transparent panel ↓ Panel decoration ↓ Button background ↓ Button icon ↓ Button text A pixel covered by all these elements may be processed repeatedly. Conceptually: GPU ↓ ┌──────────────┐ │ Background │ ← 1 │ Overlay │ ← 2 │ Panel │ ← 3 │ Decoration │ ← 4 │ Button │ ← 5 │ Icon │ ← 6 │ Text │ ← 7 └──────────────┘ Same pixel processed multiple times This is particularly relevant when your UI contains large transparent elements covering substantial portions of the screen. Unity's graphics documentation specifically recommends identifying and reducing overdraw when fill rate becomes a GPU limitation. Why Is Overdraw Particularly Important in WebGL? WebGL runs inside a browser, so performance has additional constraints compared with a native desktop application. If your application is GPU-bound, rendering unnecessary pixels means the GPU has more work to

JetBrains has described how it began centralising AI usage after development-related spending increased roughly tenfold in six months. Rather than restricting engineers to a small set of approved tools, the company built a shared access and accounting layer intended to preserve tool choice while giving teams greater visibility and control over consumption. By Matt Foster

How a branch-free loop and byte-space arithmetic let GitHub case-fold every byte of code search at >45 GiB/s on a single core. The post Don’t stop early: Case-folding source code at memory speed appeared first on The GitHub Blog .

A payroll and HR software team rebuilt three monoliths into over 120 smaller services over five years, with no dedicated migration budget. Every new feature was built as its own service instead of changing the old ones. The article covers the pull-based migration, the tools that made this possible, how costs were kept down, and the problems the team ran into along the way. By Prashanth Pasham

Most teams blame their model provider when the inference bill spikes. They're looking at the wrong line item. The real leak is architecture — and it's the difference between a token bill that scales with value and one that scales with chaos. Here's what we see shipping agentic systems in production. The hidden multiplier: agent loops A "2-minute task" is never one call. An agent fires 30–60 tool calls per run, and most frameworks stuff the entire conversation history into every prompt. So a job you'd estimate at ~4K tokens becomes 40 calls × 8K context = 320K tokens — billed at frontier rates. Frontier pricing per call looks cheap. Multiplied by agent-loop iterations, it quietly becomes the largest line in your cloud bill. The 80/20 of inference Not every call needs a frontier model. ~80% of agent traffic is routing, extraction, formatting, classification, summarization. Trivial. Leading efficient models — including top China models — handle these at near-parity. ~20% is genuine reasoning, open-ended generation, ambiguous planning. That's where frontier earns its price. Route the 80% to efficient models and reserve frontier for the 20%. Same output quality. A fraction of the bill. A unified gateway beats a drawer of API keys The trap most teams hit: they wire 4 providers with 4 clients, then let a naïve router "roam" between them. On failover it loses cache affinity, re-embeds context, and your 1.5x cost target drifts back toward ~1x — or worse. A single OpenAI-compatible endpoint across OpenAI + Gemini + leading China models fixes this: One client, one code path. Provider pinning holds cache locality; it only fails over on hard error, not price drift. Your application code never changes when you swap a model. In SEA, "PDPA-aligned" is the baseline, not a premium For Malaysia and SEA teams, inference isn't just a cost question — it's a compliance one. PDPA requires 72-hour breach notification and a designated DPO. In-region data residency (SG-hosted) is now the defa

How a face-customization puzzle at HackMIT went from clicking sliders by hand to reverse-engineering a hidden formula from 10,000 API calls. Face Value looked simple at first glance: ten sliders (Face, Skin, Hair, Brows, Eyes, Nose, Mouth, Glasses, Mole, Accessory), each 0-9, controlling a cartoon avatar. A hidden model scored every configuration, and the goal was to find one it would fully accept : Confidence ≥ 99.9% Edit distance from the starter config ≤ 5 (only half the sliders could move) Charm check: pass Sync check: pass The puzzle's own hint: "Not all features affect the model equally. Some are more sensitive than others, especially together. Single-feature sweeps can be misleading." That warning turned out to be the whole game. Phase 1: Brute Force by Hand The first instinct is the obvious one: click a slider, hit Query, read the result, adjust, repeat. Every query returned four numbers, shown together in a Reviewer panel: Probability, Charm, edit Distance, and Sync. All four had to align at once. This works, sort of. Over the first ~24 manual queries, real patterns emerged: certain Glasses values seemed to matter for Sync, Mole and Accessory nudged confidence up, some sliders had sharp peaks rather than smooth slopes. But progress plateaued hard around 60-77% confidence . Manual testing can only really explore one or two dimensions at a time, and the puzzle explicitly warned that the model cared about combinations ; you can't discover a 3-way interaction by changing one slider and squinting at the result. The first real breakthrough was small but important: after enough fiddling, one query came back with Sync: True for the first time, confidence still low (8.14%), but proof that the four conditions weren't mutually exclusive. Phase 2: Escaping the UI The turning point was popping open Chrome DevTools, clicking Query once, and grabbing the actual network request as a curl command. Underneath the slick UI was a plain JSON API: POST https://facevalue.hackmit.

A configuration change in AWS's bill computation system showed customers estimated bills in the billions and trillions of dollars for over 24 hours. AWS's own alarms detected the anomalies but failed to halt bill generation or page engineers; customer escalations alerted the company 4.5 hours later. Budget and cost anomaly alerts were disabled platform-wide during mitigation. By Steef-Jan Wiggers

The Problem with GPT-4 Pricing When I started building my AI agent hosting service, I initially planned to use OpenAI GPT-4. Then I did the math: GPT-4: ~$30 per million tokens (input) + $60 per million (output) DeepSeek Flash: ~$0.14 per million tokens That is a 200x cost difference . But Is DeepSeek Good Enough? Short answer: for most use cases, yes. I ran both models side-by-side for customer support, content generation, and code assistance. DeepSeek Flash handled 90% of tasks just as well as GPT-4. The remaining 10% (complex reasoning, nuanced writing) barely mattered for my use case. The Cache Hit Rate Secret Here is what most people miss: DeepSeek caches repeated context. With a 90% cache hit rate, the effective cost drops to ~$0.014 per million tokens. That means 100 million tokens costs about $1.40. Let that sink in. Real Numbers from My Business 24.8 billion tokens processed Total cost: ~$20 Average: $0.008 per million tokens At this rate, I can offer 100M tokens/month for $23.99 and still have 89% margin. When to Use GPT-4 Instead Be honest with yourself: Complex multi-step reasoning? GPT-4 Creative writing with specific voice? GPT-4 Everything else? DeepSeek Flash is fine The Bottom Line Do not pay 200x more for marginal quality improvement. Use DeepSeek Flash for production workloads. Save GPT-4 for the rare cases that truly need it. I run AgentChip — managed AI agent hosting powered by DeepSeek. $23.99/month with 100M tokens included.