标签：#performance

I keep seeing the same local LLM sizing mistake: "The model file is smaller than my GPU, so it should fit." That is only the first check. A 24 GB GPU does not give your model a clean 24 GB memory budget. The display stack, runtime, temporary buffers, model weights, and KV cache all compete for the same space. Here is the worksheet I use before I download a model or rent a GPU. 1. Start with the weight floor The simplest weight estimate is: weight_memory_gib = parameters * bits_per_parameter / 8 / 1024^3 For a simple 4-bit estimate: Model size Weight floor 7B 3.3 GiB 13B 6.1 GiB 70B 32.6 GiB These are floors, not promises. Real quantized files can also contain scales, metadata, and layers stored at higher precision. If you know the exact checkpoint size, use that instead of the simple bits-per-parameter estimate. Also use total parameters for a sparse mixture-of-experts model unless your runtime really offloads inactive experts. Active parameters describe compute per token. They do not automatically describe how many weights must be stored. 2. Reduce the physical capacity to a usable budget I normally start with 90 percent usable VRAM for planning: usable_vram = physical_vram * usable_fraction For a 24 GB card: 24 * 0.90 = 21.6 GiB usable The exact reserve depends on the OS, display use, driver, runtime, graph capture, allocator behavior, and other processes. The important part is to stop treating the number on the box as fully available. 3. Add the KV cache The KV cache is where context length and concurrency become expensive. A useful planning formula is: kv_cache_bytes = 2 * layers * kv_heads * head_dimension * context_tokens * concurrent_sequences * bytes_per_kv_value The factor of two stores keys and values. Take a model with: 32 layers 8 KV heads 128 dimensions per head 8,192 cached tokens 1 concurrent sequence 16-bit KV values, which use 2 bytes The KV cache is about 1 GiB. Raise the context to 32,768 tokens and it becomes about 4 GiB. Keep that context and ru

Most performance advice online assumes a baseline that doesn't exist for most of the world. Fast wifi, a recent phone, a stable connection. Lighthouse scores optimized for conditions half the planet doesn't have. I build web products for businesses in Kenya. A meaningful share of my users are on 3G, sometimes 2G, often on a budget Android phone with limited storage and a browser that hasn't seen an update in a year. Here's what that actually changes about how you build. Your bundle size is a business decision, not a dev preference A 2MB JS bundle that loads instantly on your MacBook can take 15 to 20 seconds on a real 3G connection. That's not a slow load, that's a user who left before your app finished parsing. I've watched analytics confirm this directly, drop-off spikes exactly where bundle size peaks. Skeleton screens matter more than animations Every extra animated transition is more work for a weak CPU to render. I stripped most micro-interactions out of a recent build and page-perceived speed improved more than any code-splitting change I made that month. Motion is a luxury feature for people with headroom to spare. Offline isn't an edge case, it's Tuesday Connections drop mid-session constantly, not from bad code, just from the actual infrastructure. If your app throws away form state on a dropped connection, you're actively costing your users. Basic local persistence before submission became a non-negotiable for me after watching real users lose an entire booking form to a 4 second network blip. Images are still the biggest offender in 2026 Everyone optimized images years ago and moved on. They didn't. I still regularly find production sites shipping unoptimized hero images at 3 to 4MB. On a fast connection that's invisible. On the connections a huge share of the world actually uses, that single image can be the whole page load. The real point "Fast" isn't a Lighthouse score. It's whether the app actually works for the person holding the phone it's meant fo

There is a quiet moment of panic every developer knows. You hit deploy, open the live site on your phone, and wait. One second. Two seconds. Four seconds. Still a blank white screen. A while back, I was working on a Next JS application that looked fast on high speed office Wi Fi. But when tested on a spotty mobile connection, it felt painfully slow. The initial page load was clocking in at nearly 8 seconds, and our main JavaScript bundle was a bloated 1.8 megabytes. Here is how we diagnosed the bloat, cut our load times by 47 percent, and the simple performance rules every developer should know. The Investigation: Where Was the Weight Coming From? When a website is slow, our first instinct is often to blame slow backend APIs or heavy database queries. But when I ran a performance audit, the backend was not the problem at all. The front door was just jammed with too much stuff. We were making three classic mistakes: First, we were packing for a long trip on a short walk. We were loading heavy charting libraries, complex admin tables, and pop up modals the second a user landed on the home page, even if that user only came to read a single line of text. Second, giant images were being served to tiny mobile screens, hogging precious bandwidth before any interactive buttons could even load. Third, a single state update at the top of our app was causing dozens of unseen child components to recalculate and re render unnecessarily behind the scenes. The Strategy: Trimming the Fat Instead of rewriting the entire codebase from scratch, we focused on three targeted fixes. 1. Don't Load It Until They Ask For It Why force a user to download a complex analytics chart if they have not even clicked on the dashboard tab yet? We split the app into smaller, independent code chunks. Now, the user downloads only the absolute bare minimum needed to view the immediate screen. The heavy features stay on the server until the exact moment the user interacts with them. 2. Smart Asset Delivery

The service was slow. Not down, just slow: p95 latency climbing well past where users notice, requests piling up, the kind of degradation that generates support tickets instead of alerts. And the autoscaler, the whole point of which is to add capacity when a service is under strain, sat there doing nothing. The metric it was watching said everything was fine. Average CPU utilization on the tasks was hovering around 30 percent, nowhere near the scale-out threshold. The dashboard was calm. The users were not. Both were right, and the gap between them is one of the most common autoscaling traps on a container platform. This is the first article in a series on running a multi-tenant SaaS on AWS at team scale. It is about a metric that lies, quietly, by design. Why 30 percent CPU meant 100 percent busy The service was a single-threaded application. A Node.js API, in this case, but the same is true of any process that does its real work on one thread: a classic Python or Ruby worker, most single-process runtimes. A single-threaded process can, by definition, saturate exactly one CPU core. The task it was running on had four vCPUs. So the arithmetic that matters is brutally simple: one core fully pegged / four vCPUs on the task = ~25% task-average CPU At full saturation, the busiest that process can ever make the task look is about 25 percent. Add a little async I/O overhead spread across the runtime and you land around 30 percent. That is not a service with headroom. That is a service redlining on the only core it can use, while three cores sit idle and drag the average down to a number that reads as "barely working." The autoscaling policy was tracking average CPU across the task's cores. For a workload that can only ever use one of them, that average is not a measure of load. It is a measure of load divided by four. The metric was answering a different question This is the real lesson, and it is not specific to AWS or ECS. Average CPU utilization answers "how much of th

Last month, I published a benchmark showing a 1.125× speedup from block-residual caching on 4-bit FLUX . The main lesson was not the multiplier. It was that my original quality metrics had been measuring the wrong thing, and that acceleration claims often combine speed, trajectory preservation, and perceptual quality into one number. For the follow-up, I chose a stricter target: real-time autoregressive diffusion video on an Apple M5 Max , with the definition of "real time" frozen before results were visible. The tested configuration did not meet that target. The fastest claim-eligible result was 1.418 native generated frames per second , compared with a 16 FPS target. That is an 11.28× gap . I am publishing the result because the measured bottleneck, one systems improvement, and two rejected hypotheses are useful even without a real-time result. The evidence can be checked from a repository checkout: git clone https://github.com/kkjcodes/liveframe cd liveframe python -m pip install liveframe liveframe verify \ artifacts/liveframe-publication-claims.v1.json \ --artifacts-root . liveframe recompute \ artifacts/liveframe-publication-claims.v1.json The setup LiveFrame evaluates Wan2.1-T2V-1.3B-based causal video models across NVIDIA H100 CUDA and Apple M5 Max MLX/Metal. The experiments include: Causal Forcing++ for the clean M5 performance fixture Rolling Forcing for the CUDA-to-MLX portability study Frame-wise Causal Forcing++ for the H100 cache-reuse experiment The clean M5 fixture produces 81 pixel frames at 480×832, corresponding to 5.06 seconds at the model's native 16 FPS. Before holdout results were visible, the relevant protocols froze their prompts, seeds, content strata, horizons, thresholds, aggregation rules, and stop rules. For the cross-runtime experiment, stochastic inputs were serialized once as BF16 tensors. CUDA and MLX consumed byte-identical tensors rather than relying on nominally matching random seeds. LiveFrame separates four claim layers: Numeri

Magento 2's full page cache is one of its strongest performance features — when it works. But every week, we see stores where a simple product save triggers a 30-second Varnish flush and subsequent cache stampede. The culprit is almost never Varnish itself. It's cache tags. This post covers how Magento 2 cache tags work, why broad tags destroy performance, and exactly how to audit and fix them. How Cache Tags Work in Magento 2 Every cached page, block, and data fragment in Magento is tagged with identifiers. When a product changes, Magento invalidates all cache entries tagged with that product's ID. The tag system is hierarchical: cat_p_123 — specific product cat_p — all products cat_c_5 — specific category cat_c — all categories cms_b_about_us — a CMS block cms_p — all CMS pages These tags are stored alongside cached content and used during invalidation. When you call $cache->clean(["cat_p_123"]) , every cache entry tagged with cat_p_123 is removed. This is elegant until someone tags a global block with cat_p , and saving any product flushes half your store. The Invalidation Storm Problem Here's what happens during a storm: Admin saves a simple product update (price change) Magento generates the invalidation list: cat_p_456 , cat_c (because the product is in categories), cat_p (from a badly written block) cat_p is too broad — it matches the product list page, layered navigation, homepage widgets, and every product detail page Varnish receives 50,000 BAN requests Store goes from sub-100ms response times to 2-5 seconds for the next 10 minutes while the cache rebuilds We've seen this on a store with 80,000 SKUs. A single product save dropped cache hit rate from 94% to 12%. Diagnosing Bad Cache Tags Check Your Current Tags Add this to any block template to inspect what tags are being applied: $block -> getCacheKeyInfo (); // Or for the full page: $block -> getIdentities (); For a full audit, intercept cache writes in development: // In di.xml: < type name = "Magento\Fr

Martin Spier explains how agentic workflows dramatically increase code change volume at OpenAI. He discusses the hidden systemic performance costs of rapid shipping beyond GPUs, and shares how deploying always-on AI agents automates profiling, regression detection, and continuous optimization to maintain product speed and scalability at massive global scale. By Martin Spier

While contributing to Grafana, I picked up a memory issue in the Time Comparison feature — a follow-up to earlier performance work I had done in the same area. A comparison panel was consuming significantly more memory than expected. The interesting part: the extra memory wasn't coming from real data. This post covers how I traced it to the root cause and fixed it. Background Time Comparison overlays an earlier period onto the current one — for example, this week vs. last week. The comparison data is fetched from the earlier window and shifted forward before rendering: Query → DataFrame → Prepare frame → Shift → Render │ └─ Gap filling The important detail: gap filling ran before the comparison frame was shifted. The Problem I reproduced the issue with: Parameter Value Series 500 Window 6h Interval 20s Compare offset 24h A single-period panel contained roughly 540,000 points , so a comparison panel should be about 2× the baseline . Instead, the compare frame contained 3,240,500 points — ~6× the baseline — and consumed 76.4 MB . The question was: where did the extra points come from? Investigation I first verified the baseline to rule out the query returning unexpected data. It was correct. Then I used a reproducible browser harness and a heap snapshot to inspect the extra memory. Most of it was null rows introduced during gap filling — not real samples, not copies. Following the frame through the preparation pipeline revealed why. When gap filling ran, the compare frame still represented data 24 hours in the past , but the gap-filler was using the current time range as its reference: Compare frame Current range [===== 6h =====] [===== 6h =====] └─────────────── 24h ───────────────┘ gap-filler reads this offset as one gap At a 20-second interval, 24 hours is: 24 × 60 × 60 / 20 = 4,320 intervals So up to 4,320 null positions per series were introduced purely because the frame hadn't been shifted yet. The frame was then shifted forward, leaving most of that padding out

I maintain Fulmine , a drop-in replacement for Express 5 that runs on µWebSockets.js instead of node:http . One line changes: const express = require ( " fulmine.js " ); // instead of require("express") Your middleware keeps working: helmet , cors , passport , morgan , multer , express-session and the rest. The numbers are not mine Benchmarks published by a project about itself deserve suspicion, so let me use somebody else's. HttpArena runs every framework on the same 64-core machine, in containers, under the same rules, and publishes the results. Express and Fastify are on that board too. Requests per second, from their published runs: Profile Fulmine Express Fastify Baseline (query parsing) 1,220,308 607,777 711,263 JSON (dataset + serialization) 1,111,187 395,361 522,201 Short-lived connections 1,026,789 278,163 298,779 Pipelined 7,259,814 1,009,543 1,671,338 Mixed API workload, 16 CPUs 126,282 67,724 75,633 Async Postgres 222,701 169,687 179,169 Upload (20 MB body) 2,154 2,104 1,902 That is 2.0x Express on the baseline, 2.8x on JSON, 3.7x on short-lived connections, 7.2x pipelined , and 1.9x on the mixed API profile. Against Fastify, on the same board, it is 1.7x on the baseline and 2.1x on JSON. Now the honest parts, which matter as much as the table. Look at the upload row: 1.02x. A 20 MB body is memory bandwidth and syscalls, not framework code. Everywhere the cost belongs to a library both servers call, the difference disappears: JSON.parse , zlib, OpenSSL. Speed comes from the framework only where the framework is doing the work. My entry runs in the arena's "tuned" mode, Express's and Fastify's run in "standard". On two profiles I left out of the table, static files and compressed JSON, that difference is decisive, because tuned mode allows hand-written compression and negotiation. Those rows would show 23x and 8x, and they would be measuring my entry's tuning, not the framework. I would rather not quote them. Where the speed comes from Not from one trick

A slow AI feature does not feel smart. It feels broken. That is the uncomfortable truth many AI SaaS builders hit after the demo works. The prototype answers well, the agent can call tools, and the RAG pipeline looks impressive. Then real users arrive. Prompts get longer. Queues form. Streaming starts late. One tenant uploads huge documents. Another runs bulk jobs at noon. Suddenly the same workflow that felt magical in testing feels like a spinner with an invoice attached. The fix is not simply “use a faster model.” You need an LLM latency budget : a small set of rules that says how fast each AI workflow must feel, how many tokens it can spend, when to stream, when to cache, when to route to another model, and when to stop before cost and latency drift together. This guide is for solo SaaS developers, micro SaaS builders, and AI SaaS teams shipping production features with LLM APIs, RAG, agents, or self-hosted models. Why latency budgets matter now AI platform news points in the same direction: builders are moving from chat demos to production workflows. Agent tools, web context APIs, voice agents, coding assistants, and RAG platforms are all getting more capable. At the same time, inference cost and reliability are under pressure. Latency is now a product metric. Inference efficiency is becoming a business metric. Yet many articles stop at TTFT, TPOT, quantization, batching, or model serving. Fewer show how a SaaS builder turns those ideas into a product-level budget with code, dashboards, fallbacks, and customer-safe limits. The simple model: TTFT, TPOT, and total time You do not need a PhD in serving systems to start. Track three numbers. Time to First Token Time to First Token (TTFT) is the delay between the user action and the first streamed token. It includes network time, queue time, provider overhead, tool setup, retrieval, and the model’s prefill phase. High TTFT is why a chat box feels dead. Time Per Output Token Time Per Output Token (TPOT) is the averag

When you open a long page — a news feed, an admin table, a documentation article — the browser lays...

It was 2 AM, and the on-call chat was on fire again: the order service was healthy on every dashboard, but throughput had flatlined at ~800 req/s while P99 climbed past 4 seconds. The usual suspect? A thread pool sized by guesswork during a late-night deploy, six months earlier. We'd hand-tuned maxThreads to "something that felt right," and it wasn't right anymore. That's the moment I started appreciating a different default: in Solon, all of those knobs ship as 0 — meaning auto , derived from your machine's actual CPU cores at runtime. You can go months without thinking about a single thread-pool property. This post walks through the five knobs that exist, how the auto-tuning math works, and the three failure modes that tell you it's time to touch them. The five knobs under the hood Solon exposes these on app.yml (all values are the documented defaults): # Minimum threads for the http server (0 = auto; also accepts fixed values like 2, or core multiples like x2) server.http.coreThreads : 0 # Maximum threads for the http server (0 = auto; also accepts fixed values like 32, or core multiples like x32) server.http.maxThreads : 0 # Idle thread timeout in ms (0 = auto) # supported since v1.10.13 server.http.idleTimeout : 0 # Is this an IO-bound service? (default true) # supported since v1.12.2 server.http.ioBound : true # Enable the virtual thread pool (default false) # supported since v2.7.3 solon.threads.virtual.enabled : false Notice what's missing: no hard-coded defaults for coreThreads or maxThreads . 0 means "figure it out from the hardware." That single decision removes a whole class of "copy-pasted tuning values" problems — the ones that were right for someone else's 32-core box and wrong for your 2-core container. CPU-bound or IO-bound: the one question that matters The auto-tuner only needs you to answer one question: is your workload CPU-bound or IO-bound? CPU-bound : the work happens entirely in CPU and memory — think a "hello world" handler that returns a s

Cuando llevamos modelos de lenguaje o IA a producción, la latencia es nuestro principal enemigo. Evaluar un modelo únicamente por su precisión ignora un factor crítico: el rendimiento computacional. En este post analizamos por qué la velocidad (medida en tokens por segundo) se ha convertido en una métrica clave de arquitectura y cómo puedes empezar a medirla. ¿Por qué importa la velocidad? Reducción de Latencia: Aplicaciones críticas (finanzas, salud, automatizaciones) no pueden esperar segundos por una respuesta. Eficiencia de Recursos: Optimizar el rendimiento disminuye el uso prolongado de GPUs, reduciendo directamente la factura cloud. Técnicas Clave: El uso de arquitecturas ligeras, cuantización y batch processing permite mantener la precisión mientras se incrementa el rendimiento. Ejemplo Práctico: Midiendo el rendimiento en Python Un enfoque inicial para medir la tasa de procesamiento de datos/tokens en tus pruebas de rendimiento: import time def medir_velocidad ( modelo , datos ): inicio = time . time () # Procesamiento del conjunto de datos o tokens respuesta = modelo . procesar ( datos ) fin = time . time () tiempo_total = fin - inicio tokens_procesados = len ( datos ) # O conteo exacto de tokens generados/procesados velocidad = tokens_procesados / tiempo_total print ( f " Tiempo total: { tiempo_total : . 2 f } s " ) print ( f " Rendimiento: { velocidad : . 2 f } tokens/segundo " ) return velocidad Tip de Arquitectura: Un objetivo de ~100 tokens/seg es una excelente referencia para sistemas que requieren interacción humana en tiempo real. Pasos sugeridos para optimizar: Benchmark inicial: Establece tu baseline de tokens/seg. Batch Processing: Agrupa solicitudes para maximizar el paralelismo. Modelos Destilados/Cuantizados: Evalúa si un modelo más pequeño satisface el caso de uso con una fracción de la latencia. 💬 Comunidad Pivelcode: ¿Qué herramientas o librerías utilizas para hacer profiling y benchmarking de tus modelos de IA? ¡Déjalo en los comentarios!

A practical calculator for turning an upload limit into a video bitrate, with enough margin for audio and container overhead. “Make this video smaller” is an open-ended request. “Make this three-minute video fit under 10 MB” is an engineering constraint. The second version sounds more precise, but a quality slider alone cannot solve it. A quality setting tells an encoder how aggressively to preserve detail. It does not directly tell us how many bytes the final file may contain. If the destination has a hard upload limit, the useful starting point is a bit budget. This article builds that calculation in TypeScript, then looks at the assumptions that make the answer less exact than the formula first appears. File Size Is Bitrate Multiplied by Time A video file contains several streams plus a container. For a simple MP4, the largest pieces are usually: the video stream; the audio stream; container metadata and indexing overhead. If we ignore overhead for a moment, the relationship is straightforward: file size in bits = total bitrate in bits per second × duration in seconds Rearranging it gives us the total bitrate available for a target size: total bitrate = target size in bits / duration in seconds That total must cover both video and audio. The approximate video budget is therefore: video bitrate = total bitrate - audio bitrate - overhead allowance The result is not a promise. It is a budget that an encoder can aim at. Be Explicit About MB and MiB Before writing code, decide what “10 MB” means. Storage vendors and many web services use decimal megabytes: 1 MB = 1,000,000 bytes Operating systems and developer tools often display binary mebibytes: 1 MiB = 1,048,576 bytes The difference is about 4.9%. That is large enough to turn a file that looks safe locally into a rejected upload. For a hard external limit, I prefer to calculate with decimal MB and keep an additional safety margin. For an internal tool where the unit is clearly MiB, I make that choice explicit in th

Originally published on tamiz.pro . MongoDB aggregation pipelines are powerful tools for processing and transforming data directly within the database. However, when dealing with large datasets, poorly optimized pipelines can become a significant performance bottleneck. This deep-dive explores advanced strategies and best practices to ensure your large-scale MongoDB aggregation pipelines run efficiently and effectively, transforming raw data into actionable insights without grinding your system to a halt. Table of Contents Understanding the Aggregation Pipeline Lifecycle The Critical Role of Indexing Indexes for $match and $sort Stages Compound Indexes and Covered Queries Partial Indexes for Specific Workloads Strategic Stage Ordering Pushing $match and $project Early Leveraging $sort and $limit Together Memory Management and Disk Spills allowDiskUse and its Implications Strategies to Minimize Disk Spills Leveraging the Query Optimizer and Explain Plan db.collection.explain() Interpreting Explain Plan Output Sharding Considerations for Aggregations Shard Key Design for Aggregation Workloads Targeted vs. Broadcast Aggregations Advanced Optimization Techniques Using $lookup for Joins and its Performance Impact Optimizing $group Stages Batching and Incremental Aggregations Production Best Practices Frequently Asked Questions Understanding the Aggregation Pipeline Lifecycle Before diving into optimizations, it's crucial to understand how MongoDB processes aggregation pipelines. An aggregation pipeline is a sequence of stages that process documents from a collection. Each stage performs an operation on the input documents and outputs a stream of documents to the next stage. This stream-based processing is key to its efficiency, but it also means that the output of one stage directly impacts the performance of subsequent stages. The MongoDB query optimizer attempts to reorder certain stages for efficiency, but it's not omniscient. Your strategic design choices profoundly

Introduction React is fast, but that doesn't mean every React application is. One of the most common performance problems—especially in growing applications—is unnecessary re-rendering . A small project with a few components may feel instant, but as your application grows, unnecessary renders can cause sluggish interfaces, input lag, excessive CPU usage, and poor user experience. The good news is that unnecessary re-renders are usually preventable once you understand why React re-renders components . In this article, we'll explore how React rendering works, learn how to identify performance bottlenecks, and apply practical optimization techniques such as React.memo , useMemo , useCallback , better state management, and component architecture. Whether you're building dashboards, e-commerce stores, SaaS products, or portfolio websites, these techniques will help you write more efficient React applications. Table of Contents Understanding React Rendering What Causes Unnecessary Re-renders? Identifying Performance Problems Optimizing with React.memo Optimizing Expensive Calculations with useMemo Preventing Function Recreation with useCallback State Colocation Splitting Components Optimizing Context Rendering Large Lists Using the React Profiler Best Practices Common Mistakes Performance Tips Security Considerations Accessibility Considerations SEO Considerations Real Project Example Conclusion Discussion Background Before optimizing anything, it's important to understand what React actually does. A render simply means React executes your component function to determine what the UI should look like. That does not always mean the browser updates the DOM . React compares the new Virtual DOM with the previous one and only updates the parts that actually changed. However, if many components re-render unnecessarily, React still has to: Execute component functions Recreate objects Recreate arrays Recreate event handlers Compare Virtual DOM trees All of that work adds up. Step

Your GitOps Hub Will Become the Bottleneck Long Before Cluster Count Tells You GitOps hub bottlenecks are usually predicted more accurately by watched object volume, reconcile queue depth, and controller memory growth than by cluster count alone. Large-scale testing described in the discussion showed Argo CD application controllers hitting out-of-memory failures around 15,000 to 20,000 cached objects per hub, while sharding and tuning delayed the limit without removing the underlying memory cost. The most important lesson was uncomfortable because it challenged the usual instinct to keep tuning the existing platform. At very large scale, architecture mattered more than configuration. Hydrated manifests helped. More replicas helped. Dynamic sharding helped. None of them changed the fact that a centralized reconciliation model still had to hold and process a huge amount of state. The testing was not presented as a universal benchmark or as proof that one tool always beats another. It was a record of one setup, built through dozens of iterations over several months, and the failures were as valuable as the successful runs. That is exactly why the results matter. They show where teams should look before the hub becomes the thing taking the fleet down. Cluster count is the wrong first metric A fleet with 1,000 tiny clusters may place less pressure on a GitOps control plane than a much smaller fleet containing thousands of applications and deeply expanded resource trees. The number of managed clusters is visible and easy to report, but it does not describe the controller’s actual workload. The more useful mental model is objects over clusters. Each application contributes desired state, live state, cached trees, reconciliation work, and queue activity. A cluster that runs a few small addons may be cheap to manage. Another cluster with many applications and large manifest sets may consume far more memory and reconciliation time. That means two fleets with the same cluster

TimescaleDB 2.27, released May 12 2026, extends bloom-filter batch pruning from reads to writes. UPDATE, DELETE, and UPSERT against compressed columnstore data can now skip decompressing batches that provably cannot contain the target rows. The reported gains are real: up to 160x for selective UPDATE/DELETE, and over 2x for UPSERT. The feature is automatic. Whether it is actually firing on your workload is not something you can assume, and the only way to confirm it is to read new EXPLAIN counters that the release notes mention but do not explain. Worse, the counter names are inconsistent between the write paths, so even a careful reader ends up guessing. This post is about reading those counters correctly, and about the two things in this release that will silently break a query if you upgrade without noticing them. What is actually being skipped A quick model of the mechanism, because the counters only make sense against it. Hypercore stores compressed data in batches, roughly a thousand rows each. For columns that are not the segmentby key, TimescaleDB maintains a sparse bloom filter per batch: a small probabilistic summary that answers one question, "could this batch contain column = X ?", without touching the compressed payload. A bloom filter has a useful asymmetry. A negative is certain: if the filter says no, the value is definitely absent, and the batch can be skipped whole. A positive is not: the filter says "maybe", you decompress, and sometimes the value is not there after all. That last case is a false positive, and it is the number that tells you whether the whole scheme is paying off. Before 2.27, a DELETE ... WHERE sensor_id = 'x' against compressed data decompressed every candidate batch to check. Now the bloom filter is consulted first, and batches that cannot match are never decompressed. The work you save is the decompression of the batches that get pruned. The work you waste, when the filter is poorly matched to your data, is the bloom check on

Idempotency, vendor failure, monitoring that catches the invisible outages, and the tradeoffs nobody warns you about, lessons from scaling payments infrastructure. Most software fails quietly. A page renders slowly, a recommendation is a little off, a report is stale by an hour. Users shrug and move on. Payments doesn't work like that. When payments break, someone's money is in a place neither of you can account for, and the clock starts ticking on their patience. There's no graceful degradation. Either the money moved, or it didn't, and someone needs to know which. I've spent a good chunk of my career building and scaling payments infrastructure, and it has quietly rewired how I think about engineering in general. Here's what stuck. 📋 The short version # Lesson One-line summary 1 Idempotency You will receive the same request twice. Design for it. 2 Vendor failure Gateways are vendors. Ask "when," not "if." 3 Monitoring Never learn about an outage from a customer. 4 The unglamorous stuff Ledgers, reconciliation, state machines, refunds. 5 Tradeoffs Every lesson above fights at least one other. 1. 🔁 Idempotency isn't a feature. It's a foundation. The first hard lesson: you will receive the same request twice. Not "might." Will. A client times out waiting for your response and retries. A user double-taps a button on a bad connection. A queue consumer crashes after processing but before acknowledging. A gateway sends the same webhook four times because it never got a 200 back. None of these are exotic failure modes, they're Tuesday. If your system treats every incoming call as a new instruction, every one of those scenarios becomes a double charge. And a double charge isn't a bug you fix quietly in the next release. It's a support ticket, a refund, a reconciliation entry, and a customer who now checks their statement every time they use you. The fix is conceptually simple and operationally demanding: every operation that moves money must be uniquely identifiable and sa