标签：#Performance

Your GitOps Hub Will Become the Bottleneck Long Before Cluster Count Tells You GitOps hub bottlenecks are usually predicted more accurately by watched object volume, reconcile queue depth, and controller memory growth than by cluster count alone. Large-scale testing described in the discussion showed Argo CD application controllers hitting out-of-memory failures around 15,000 to 20,000 cached objects per hub, while sharding and tuning delayed the limit without removing the underlying memory cost. The most important lesson was uncomfortable because it challenged the usual instinct to keep tuning the existing platform. At very large scale, architecture mattered more than configuration. Hydrated manifests helped. More replicas helped. Dynamic sharding helped. None of them changed the fact that a centralized reconciliation model still had to hold and process a huge amount of state. The testing was not presented as a universal benchmark or as proof that one tool always beats another. It was a record of one setup, built through dozens of iterations over several months, and the failures were as valuable as the successful runs. That is exactly why the results matter. They show where teams should look before the hub becomes the thing taking the fleet down. Cluster count is the wrong first metric A fleet with 1,000 tiny clusters may place less pressure on a GitOps control plane than a much smaller fleet containing thousands of applications and deeply expanded resource trees. The number of managed clusters is visible and easy to report, but it does not describe the controller’s actual workload. The more useful mental model is objects over clusters. Each application contributes desired state, live state, cached trees, reconciliation work, and queue activity. A cluster that runs a few small addons may be cheap to manage. Another cluster with many applications and large manifest sets may consume far more memory and reconciliation time. That means two fleets with the same cluster

TimescaleDB 2.27, released May 12 2026, extends bloom-filter batch pruning from reads to writes. UPDATE, DELETE, and UPSERT against compressed columnstore data can now skip decompressing batches that provably cannot contain the target rows. The reported gains are real: up to 160x for selective UPDATE/DELETE, and over 2x for UPSERT. The feature is automatic. Whether it is actually firing on your workload is not something you can assume, and the only way to confirm it is to read new EXPLAIN counters that the release notes mention but do not explain. Worse, the counter names are inconsistent between the write paths, so even a careful reader ends up guessing. This post is about reading those counters correctly, and about the two things in this release that will silently break a query if you upgrade without noticing them. What is actually being skipped A quick model of the mechanism, because the counters only make sense against it. Hypercore stores compressed data in batches, roughly a thousand rows each. For columns that are not the segmentby key, TimescaleDB maintains a sparse bloom filter per batch: a small probabilistic summary that answers one question, "could this batch contain column = X ?", without touching the compressed payload. A bloom filter has a useful asymmetry. A negative is certain: if the filter says no, the value is definitely absent, and the batch can be skipped whole. A positive is not: the filter says "maybe", you decompress, and sometimes the value is not there after all. That last case is a false positive, and it is the number that tells you whether the whole scheme is paying off. Before 2.27, a DELETE ... WHERE sensor_id = 'x' against compressed data decompressed every candidate batch to check. Now the bloom filter is consulted first, and batches that cannot match are never decompressed. The work you save is the decompression of the batches that get pruned. The work you waste, when the filter is poorly matched to your data, is the bloom check on

Idempotency, vendor failure, monitoring that catches the invisible outages, and the tradeoffs nobody warns you about, lessons from scaling payments infrastructure. Most software fails quietly. A page renders slowly, a recommendation is a little off, a report is stale by an hour. Users shrug and move on. Payments doesn't work like that. When payments break, someone's money is in a place neither of you can account for, and the clock starts ticking on their patience. There's no graceful degradation. Either the money moved, or it didn't, and someone needs to know which. I've spent a good chunk of my career building and scaling payments infrastructure, and it has quietly rewired how I think about engineering in general. Here's what stuck. 📋 The short version # Lesson One-line summary 1 Idempotency You will receive the same request twice. Design for it. 2 Vendor failure Gateways are vendors. Ask "when," not "if." 3 Monitoring Never learn about an outage from a customer. 4 The unglamorous stuff Ledgers, reconciliation, state machines, refunds. 5 Tradeoffs Every lesson above fights at least one other. 1. 🔁 Idempotency isn't a feature. It's a foundation. The first hard lesson: you will receive the same request twice. Not "might." Will. A client times out waiting for your response and retries. A user double-taps a button on a bad connection. A queue consumer crashes after processing but before acknowledging. A gateway sends the same webhook four times because it never got a 200 back. None of these are exotic failure modes, they're Tuesday. If your system treats every incoming call as a new instruction, every one of those scenarios becomes a double charge. And a double charge isn't a bug you fix quietly in the next release. It's a support ticket, a refund, a reconciliation entry, and a customer who now checks their statement every time they use you. The fix is conceptually simple and operationally demanding: every operation that moves money must be uniquely identifiable and sa

Engineering posts often end with: The new design is correct, scalable, and fast. Fast compared with what? When we changed Podium so tied players rank by arrival time instead of player ID, we added: a Lua script; a per-leaderboard sequence; a public-ID mapping; a second sorted set for ascending order. That design is fairer. It is also impossible for it to be free. So we built two benchmark layers: direct Redis strategy benchmarks to isolate the data-model cost, and end-to-end HTTP benchmarks to show what users actually experience. We are publishing the results, including the regression, because performance claims are useful only when readers can inspect the workload and reproduce the measurement. TeneficGames / podium High-performance, Redis-backed leaderboards for games and competitive applications. Podium High-performance, Redis-backed leaderboards for games and competitive applications. Podium provides ready-to-run HTTP and gRPC APIs for scores, ranks, seasons, and player-relative views. It is designed for backend teams operating large fleets of independent leaderboards without provisioning each leaderboard in advance. Fair, deterministic ordering when scores are equal. Single and bulk score updates, including multi-leaderboard fan-out. Standalone Redis and real Redis Cluster integration coverage. Deploy one multi-architecture OCI image with Docker, containerd, Kubernetes or another OCI-compatible runtime. Quickstart · Performance · API · Documentation · Helm chart · Docker Hub · GHCR Quickstart Start Redis 8.2 and the latest stable Podium image: docker network create podium docker run --detach --name podium-redis --network podium redis:8.2-alpine docker run --detach --rm --name podium \ --network podium \ --publish 8880:8880 \ --publish 8881:8881 \ --env PODIUM_REDIS_HOST=podium-redis \ --env PODIUM_REDIS_PORT=6379 \ trungdlp/podium:latest start Verify the service: curl http://localhost:8880/healthcheck WORKING Submit two equal scores: curl --request … View on Gi

When building modern Python applications—whether scraping web pages, fetching data from external APIs, or querying databases—IO-bound operations often slow down execution. Python’s concurrent.futures module provides a high-level, elegant interface for running tasks asynchronously. In this guide, we'll break down what Futures are, why you need them, and how to use them effectively using a practical e-commerce product service. What is a Future? A Future represents an eventual result of an asynchronous operation. When you launch an expensive, long-running task concurrently, your program doesn't pause to wait for the output. Instead, it instantly gets back a Future object —a low-cost proxy or standard "claim ticket." The Future acts as a placeholder for a result that hasn't been computed yet. It keeps track of the task's execution state ( PENDING , RUNNING , CANCELLED , or FINISHED ). Once the task finishes, the Future stores the return value or any exception thrown during execution. Why are Futures Needed? In standard synchronous Python execution, calling a function blocks your main thread until that function finishes: Task 1 (2s) ──> Task 2 (3s) ──> Task 3 (1s) = 6 seconds total When dealing with IO-bound operations (like waiting for network responses or reading disks), your CPU sits completely idle during those delays. By offloading tasks into background threads or processes via Futures, your application can run multiple IO operations simultaneously: Task 1 (2s) [████████] Task 2 (3s) [████████████] Task 3 (1s) [████] ----------------------------------------- Total Time: 3 seconds (time of longest task) When Should You Use Futures? IO-Bound Workloads: Scraping multiple web pages, batch-calling microservices, querying multiple databases, or fetching images concurrently ( ThreadPoolExecutor ). CPU-Bound Parallelism: Performing heavy mathematical operations or image processing across multiple CPU cores ( ProcessPoolExecutor ). Decoupled Workflows: When you want to trigg

Embedding a browser game looks simple: <iframe src= "https://games.example.net/my-game" ></iframe> That line lets a third party join the page lifecycle immediately. It can download a large bundle, establish connections, run scripts, request storage, display advertising, or fail before the visitor decides to play. AI-assistance disclosure: I used AI to help draft and edit this article, then reviewed its architecture, code, claims, and limitations before publication. For a game directory, that default is both expensive and surprising. A visitor may have opened the page to read the controls, compare games, or check whether the game works on a phone. Loading the player before that intent is known wastes bandwidth and collapses two separate decisions—visiting the guide and opening the third-party game—into one. While working on a browser-game portal, I treated the site and the embedded player as two different trust and performance boundaries. The page renders first-party information immediately. The third-party frame is created only after an explicit Play action. This article explains that pattern and the engineering details that made it useful rather than merely decorative. Start with a two-layer model The outer page should be a complete page without the game: A descriptive heading and summary Controls and gameplay tips Developer and platform information Related games and category navigation A poster or cover image A real button that starts the player The inner layer is a small launcher responsible for the game lifecycle: Validate the requested game. Wait for an intentional Play action. Create the provider iframe. Report loading state. Offer recovery when loading is slow or blocked. Remove the frame when the player resets it. Do not put the remote URL in the initial markup Native iframe lazy loading is helpful below the fold, but it is not an intent gate. Browsers decide when a loading="lazy" frame is close enough to fetch. If the goal is “no third-party game request be

When building local RAG (Retrieval-Augmented Generation) applications, edge agents, or serverless AI pipelines, developers usually hit a wall with standard vector stores: memory overhead. Running a dedicated vector database locally often demands hundreds of megabytes—or gigabytes—of RAM just to keep indices warm. On the flip side, lightweight local options like scanning raw JSON files or querying SQLite don't scale well when vector dimensions climb into the thousands (1536d+). We built Bitweave to solve this exact trade-off: a zero-copy, SIMD-accelerated hybrid retrieval engine in Rust (with Python bindings) that handles categorical filtering and vector search while locking its active heap footprint under 1.1 MB RSS. The Architecture: How Bitweave Achieves Sub-Millisecond Speed at <1.1 MB RAM Bitweave relies on a 3-part design to maximize search speed while keeping memory consumption negligible: [ Categorical Filters ] ---> Bit-Sliced Bitmaps │ ▼ [ Query Vector (1536d) ] --> 1-Bit SIMD Pre-Filtering (Hamming Distance) │ (Top K Candidates) ▼ [ Raw Embeddings Buffer ] -> Zero-Copy Float32 Rescoring (exact_rescore=True) │ ▼ Top-K Results Array (NumPy) Zero-Copy Memory Mapping (memmap2) Instead of deserializing index files into Python RAM or Rust heap space, Bitweave uses memory-mapped files (.bweave). The operating system's page cache handles lazy loading of index segments directly from disk into virtual address space. As a result, the active RSS memory footprint remains static around 1.1 MB, whether your index holds 5,000 or 200,000 records. 1-Bit Vector Quantization & SIMD Hamming Distance High-dimensional float32 vectors (1536d) are quantized down to 1-bit sign masks (where values > 0 map to 1 and <= 0 map to 0). During pre-ranking, Bitweave uses SIMD bitwise XOR and POPCNT operations to compute Hamming distances across candidate vectors in microseconds. Zero-Copy 2-Pass Float32 Rescoring (exact_rescore=True) Quantization speeds up initial candidate selection, but f

log.info("user logged in") looks free. It isn't. Behind that one line is a chain of decisions — buffer or not, flush or not, block or drop, same thread or another — and each one trades latency , throughput , and durability against the others. This post walks the whole chain, from the method call down to the bytes hitting the disk platter. If you've ever wondered why your p99 latency has a mysterious spike, why logs vanish after a crash, or what "async logging" actually buys you, this is for you. First, the map: facade vs. implementation Java logging is a two-layer cake, and mixing up the layers is the #1 source of confusion. The facade is the API your code calls. The implementation is what actually writes the bytes. your code │ log.info(...) ▼ ┌───────────────────────────────┐ │ Facade: SLF4J (or Log4j2 API)│ ← the interface you compile against └──────────────┬────────────────┘ │ bound at runtime ┌───────────┼────────────┬──────────────┐ ▼ ▼ ▼ ▼ Logback Log4j2 Core java.util.logging ... (the engine that buffers, formats, and flushes) SLF4J — the de-facto standard facade. Your app should log against this. Logback — the reference SLF4J implementation. Solid, widely deployed. Log4j2 — the performance-focused implementation, famous for its lock-free async loggers. java.util.logging (JUL) — built into the JDK, rarely chosen on purpose. Why the split? So you can swap engines without touching a single log. call. Everything interesting in this post — the buffering, the flushing, the async magic — happens in the implementation layer. The anatomy of a single log call Before we talk flushing, let's see what one log.info(...) actually does. There are five stages: 1. Level check → is INFO enabled for this logger? (cheap, often the fastest bail-out) 2. Build LogEvent → capture message, timestamp, thread, MDC context, maybe a stack trace 3. Filter → run any configured filters 4. Layout / encode → turn the event into bytes ("2026-07-28 12:00:01 INFO ...") 5. Append → write those by

I run HydraDNS, an open-source DNS security gateway in Go. Last month I sat down to find out what one box could actually handle before I put it on anyone else's network. The plan had a rule I'd written for myself: every number we discover becomes either a sales claim or a fix ticket. No number, no claim. I expected to find one bottleneck. I found two, stacked on top of each other, and a third thing I wasn't looking for: a data structure in our own documentation that had never existed in the code. Everything below was measured on a 22-core dev machine with load generated inside the container, using dnspyre, so docker-proxy and host networking stay out of the numbers. It's not appliance hardware and I'm not making appliance claims. The shapes are what matter. The first ceiling: ~500 QPS, and it didn't care what I threw at it The first redline run capped at roughly 500 queries per second. Fine, servers have limits. What made it interesting was that the cap didn't move. Blocked queries: ~500. Cached queries that never touch upstream: ~500. Two code paths that do completely different work, hitting the same wall, with the CPU sitting under 30% of 22 cores. That signature is worth memorizing. When two very different paths hit the same ceiling and the CPU is bored, the bottleneck isn't in either path. It's in something they share, or something upstream of both. Ours was in the blocklist check. IsBlocked ran a SQL COUNT against a 92k-row blocklist_entries table on every query . Not just candidate blocks, every query, because the check sits in front of the cache, so even cache hits paid for it. And all of those reads were serialized through a single SQLite connection, MaxOpenConns=1 , which was also absorbing the async write traffic from query logging. The engine's self-measured latency under load: p50 of 50ms, p99 of 5000ms. Five full seconds at the tail, for DNS, which is supposed to be the fast part of the internet. The part where I found out our docs were lying Here's the

How a Missing Database Index Turned an 8ms API into an 8-Second Nightmare Every developer...

600 MiB of headroom My RTX 4070 was running Qwen 35B beautifully after the --cpu-moe trick from a previous run. The tokens/sec were where I wanted them. VRAM sat at 11,714 MiB out of 12,281 — 95% full. That leaves 600 MiB. Not enough for a serious agent. The context window I was giving llama.cpp was -c 4096 . Fine for chat. Not fine when a Claude Code-style agent hands the model 12,000 tokens of tool definitions before it says hello. I wanted -c 32768 . That's an 8× jump. And the memory that grows with context length is the KV cache. Multiply the cache by 8 with 600 MiB free, and llama.cpp dies during warm-up. I know because I tried it first. What actually sits on the GPU After offloading the MoE experts to CPU (the previous chapter's trick), the GPU is holding two things: The attention weights and non-MoE parameters The KV cache — a running record of every token the model has already read The first is fixed. The second grows linearly with context length. Double the context, double the cache. -c 4096 → -c 32768 doesn't just want 8× more tokens processed, it wants 8× more cache resident in VRAM the whole time. There is no room. So the cache itself has to shrink. Two flags llama.cpp takes two flags for KV cache dtype: llama-server -m qwen35.gguf -ngl 99 --cpu-moe -c 32768 \ -ctk q8_0 -ctv q8_0 -ctk is the Key cache, -ctv is the Value cache. Default is f16 (16-bit). q8_0 cuts each in half. Halving both means the KV cache footprint drops by roughly 50%. That freed-up VRAM is exactly what I need to make the context 8× bigger without touching the model weights. The measurement Same prompt, same seed, two runs — one at f16 KV, one at q8_0 KV: KV dtype Max -c I could allocate Tokens/sec (decode) Perplexity delta f16 (default) 4096 ~34.6 baseline q8_0 32768 ~34.1 negligible in my tests The speed loss is inside noise. The context is 8× longer. The quality drop I could not tell apart from run-to-run variance. Community measurements agree: symmetric q8_0 KV lands somewhere unde

Performance optimization in React is often surrounded by myths. Developers routinely wrap every single component in React.memo , wrap every function in useCallback , and wonder why their application is still sluggish or memory-heavy. Premature optimization can actually degrade app performance and clutter your codebase. To build fast React applications, you need techniques that address actual bottlenecks: unnecessary re-renders, unoptimized state placement, oversized bundles, and main-thread blocking. Here are five practical React performance optimization techniques that deliver measurable results in production. 1. Push State Down (Fix Rerender Cascades) Before reaching for useMemo or React.memo , evaluate your state placement . When state lives too high up in the component tree, every state update forces the entire sub-tree to re-render. ❌ The Anti-Pattern: State at the Root // Changing `color` forces <HeavyChartComponent/> and <ComplexTable/> to re-render! export default function App () { const [ color , setColor ] = useState ( ' #6366f1 ' ); return ( < div > < input type = "color" value = { color } onChange = { ( e ) => setColor ( e . target . value ) } /> < p style = { { color } } > Sample Text </ p > < HeavyChartComponent /> < ComplexTable /> </ div > ); } ✅ The Fix: Component Isolation Move the isolated state and its control into its own dedicated child component: Javascript function ColorPicker () { const [ color , setColor ] = useState ( ' #6366f1 ' ); return ( < div > < input type = "color" value = { color } onChange = { ( e ) => setColor ( e . target . value ) } /> < p style = { { color } } > Sample Text </ p > </ div > ); } export default function App () { return ( < div > < ColorPicker /> { /* These components are no longer impacted by color state changes */ } < HeavyChartComponent /> < ComplexTable /> </ div > ); } 2. Pass Components as Children (Component Composition) Sometimes state must remain in a parent component, but you don't want child components

SQL is quite flexible, making it easy to write a single query that works for two situations: one without a parameter and a WHERE clause, and another with a parameter for filtering, all in the same SQL query. For example, I came across a benchmark comparing MongoDB and PostgreSQL that shows how to handle pagination effectively—by avoiding OFFSET and instead using the last value to fetch the next set of results. The first page includes a WHERE clause along with ORDER BY and LIMIT, while the following pages add an extra WHERE condition. In the MongoDB version of this benchmark, the filter is handled within the application, which leads to two separate queries for these scenarios. export async function getOrders ( cursor ) { const match = cursor ? { timestamp : { $gt : new Date ( cursor ) } } : {}; const rows = await orders . aggregate ([ { $match : match }, { $sort : { timestamp : 1 } }, { $limit : PAGE_SIZE }, ]) We can do the same in PostgreSQL using a single prepared statement. SQL is such a powerful language that it often feels tempting to write it this way: SELECT * FROM orders WHERE $ 1 :: timestamptz IS NULL OR "timestamp" > $ 1 ORDER BY "timestamp" ASC LIMIT $ { PAGE_SIZE } If $1 is NULL, it skips the second condition in the OR clause and retrieves all rows without filters, resulting in a broad fetch. When $1 has a value, it filters the results using that specific value, enabling a more targeted search. However, using a generic query can sometimes lead to a less-than-ideal execution plan that's not perfectly tailored for each specific situation. I gave it a try: drop table if exists orders ; create table orders ( order_id text primary key , "timestamp" timestamptz not null ); create index idx_orders_timestamp on orders ( "timestamp" ); insert into orders select 'ORD-' || g , '2025-01-01' :: timestamptz + g * interval '1 minute' from generate_series ( 1 , 5000000 ) as g ; analyze orders ; prepare getorders ( timestamptz , int ) as select * from orders where $ 1 :

Hello everyone, I am currently building a high-performance API gateway that integrates business logic components—implemented via WASI and running within Wasmtime—as HTTP/TCP/QUIC handlers. I am exploring the best design approach to achieve a zero-copy data path from the upstream network layer—specifically hyper for HTTP/1.x and HTTP/2, and h3-quinn (based on Quinn) for HTTP/3—to the wasi:http guest environment. Given that: hyper and h3-quinn each manage their own internal buffer pools (e.g., bytes::Bytes ), asynchronous read/write streams, and frame decoders. Wasmtime's wasmtime-wasi-http implements the wasi:http (WASIp2) specification, which relies on resource types such as InputStream and OutputStream . I aim to minimize memory copying and CPU overhead when passing large request bodies or streaming responses across the sandbox boundary. For those experienced with bridging I/O between the host and guest in Wasmtime, I have a few architectural questions: Buffer ownership and memory mapping: How can host-side bytes::Bytes (from hyper or h3-quinn ) be mapped or bridged into Wasm linear memory (and vice versa) without requiring a CPU-based memcpy ? Does Wasmtime's resource streaming support direct memory views, or are we essentially limited to copying data chunks via guest memory pointers? Adapting stream abstractions: hyper uses http_body_util::combinators / http_body::Body , h3 uses its own stream primitives, while wasi:http uses wasi:io/streams . What is the idiomatic way to efficiently adapt these asynchronous streams on the host side (i.e., within the wasmtime-wasi-http handler implementation) without blocking the tokio runtime? Backpressure propagation: How can backpressure signals be correctly propagated from the Wasm guest (e.g., when the guest's InputStream is consuming data slowly) all the way back to the Quinn congestion controller or Hyper connection pool, thereby avoiding unbounded buffering on the host side? If anyone has built similar high-performance ga

The Go pod is running in production. CPU limit set to 2, metrics look reasonable. But under load, P99 latencies spike intermittently with no obvious cause. No errors, no goroutine leaks, just latency blowing up on traffic bursts. The root cause is usually invisible: GOMAXPROCS equals the number of CPUs on the physical node, not the container limit. Your Go app thinks it has 32 CPUs when it only has 2. The Linux kernel handles the gap in its own way — CFS throttling. What GOMAXPROCS reads (and what it ignores) By default, the Go runtime computes GOMAXPROCS via runtime.NumCPU() , which reads the number of CPUs available at the OS level. On a 32-core Kubernetes node, that returns 32 — regardless of what resources.limits.cpu says in your pod spec. Kubernetes CPU limits are enforced through Linux cgroups (v1 or v2). Cgroups are transparent to processes: a pod with limits.cpu: "2" doesn't see two virtual CPUs, it sees all the node's CPUs and gets suspended when it consumes too much. The Go runtime, historically, never read cgroups. It trusted the physical core count. package main import ( "fmt" "runtime" ) func main () { // Inside a pod with limits.cpu: "2" on a 32-core node fmt . Println ( runtime . NumCPU ()) // → 32 fmt . Println ( runtime . GOMAXPROCS ( 0 )) // → 32 } CFS throttling: how the kernel slows you down The Linux CFS (Completely Fair Scheduler) enforces CPU limits via two cgroup parameters: cpu.cfs_quota_us (allowed CPU time) and cpu.cfs_period_us (measurement window, 100 ms by default). A pod limited to 2 CPUs gets at most 200 ms of CPU time per 100 ms window. When Go spawns 32 OS threads for 32 parallel goroutines, those threads compete for physical CPUs. Once their combined usage exceeds the cgroup quota within the current window, the kernel suspends all threads in the cgroup until the next window starts. That's throttling: a complete application freeze lasting anywhere from a few milliseconds to several tens of milliseconds. A handful of these per second

The problem with "just add more workers" Most Spark performance issues on Databricks aren't solved by scaling the cluster — they're caused by shuffle and skew , and no amount of extra nodes fixes a badly partitioned join. This post builds a realistic pipeline (order events joined against a small dimension table, aggregated, and written to Delta Lake) from the ground up, and uses it to work through: How Spark's shuffle actually behaves during a wide transformation Diagnosing and fixing data skew with salting and adaptive query execution (AQE) Laying out the resulting Delta table with Z-Ordering so downstream queries skip irrelevant files Governing access to the whole pipeline with Unity Catalog Architecture overview Pipeline shape — a batch job reading raw events, joining against a dimension table, aggregating, and writing to a governed Delta table: What happens inside a shuffle stage — this is the part most tutorials skip, and it's the key to understanding why skew hurts: Step 1 — Set up governed tables in Unity Catalog Everything downstream depends on tables being registered under Unity Catalog, which gives you centralized access control and lineage instead of per-workspace table grants. -- setup.sql, run in a Databricks SQL or notebook cell CREATE CATALOG IF NOT EXISTS retail_analytics ; CREATE SCHEMA IF NOT EXISTS retail_analytics . events ; CREATE TABLE IF NOT EXISTS retail_analytics . events . raw_orders ( order_id STRING , customer_id STRING , product_id STRING , quantity INT , event_ts TIMESTAMP ) USING DELTA LOCATION 'abfss://data@<storage-account>.dfs.core.windows.net/raw_orders' ; CREATE TABLE IF NOT EXISTS retail_analytics . events . dim_products ( product_id STRING , category STRING , unit_cost DOUBLE ) USING DELTA LOCATION 'abfss://data@<storage-account>.dfs.core.windows.net/dim_products' ; GRANT SELECT ON TABLE retail_analytics . events . raw_orders TO `analysts` ; Step 2 — Read and force a broadcast join for the small dimension table dim_products is s

Headline: Next.js Middleware (middleware.ts at the project root) runs before every matched request — before cache, before rendering, before the route. That position makes it right for auth redirects, A/B cookie bucketing, and locale detection. Wrong for database queries and heavy imports. In 2026, Middleware on Vercel runs on Fluid Compute (standard Node.js), so the constraint is latency budget, not API availability. Key takeaways Middleware runs before every matched request — before cache, rendering, or route handler — the right layer for auth, locale, and A/B bucketing. Middleware can read requests, set cookies, redirect, rewrite, or return early — without the route running. DB queries and large packages add latency to every request. On Vercel in 2026, Middleware runs on Fluid Compute (standard Node.js). The constraint is latency: every added millisecond is paid on every matched request. Use matcher to scope Middleware to only the routes that need it; without it Middleware runs on every static asset request. Auth in Middleware = verifying a self-contained JWT without a DB call. Full session validation belongs in the route. I spent a long time only using Middleware for locale redirects. After shipping auth-protected routes and an A/B test, the full shape became clear. What is Next.js Middleware and where does it run? Middleware is exported from middleware.ts at the project root. It intercepts matched requests before route resolution, cache lookup, and Server Component execution. Returns one of four types: pass through ( NextResponse.next() ), redirect, rewrite (serve different content while keeping original URL in address bar), or a direct response. export function middleware ( request : NextRequest ) { return NextResponse . next (); } export const config = { matcher : [ ' /((?!_next/static|_next/image|favicon.ico).*) ' ], }; Without matcher , Middleware runs on every request including static files. On Vercel in 2026, Middleware runs on Fluid Compute — standard Nod

Have you ever noticed that your GenAI applications are spending massive amounts of time and money re-reading the exact same setup text? Every time a user asks a short question in a chatbot, the Large Language Model (LLM) must re-read your entire 2,000-word corporate playbook, your agent's system rules, and the full chat history from scratch. This phase is called the pre-fill math phase, and it drives up both your cloud bill and your user latency (Time-to-First-Token).With Amazon Bedrock Prompt Caching for Claude 4.6 (both Sonnet 4.6 and Opus 4.6), this problem is completely solved. You can achieve up to a 90% cost reduction on input tokens and an 85% drop in latency by using a clever architectural shortcut. Here is exactly how it works under the hood, how AWS maintains it across API requests, and how to implement it using Python. The Secret Architecture: Model Inference vs. AWS Infrastructure Prompt caching is a beautiful team effort between the AI model hardware and the AWS cloud infrastructure. The Model Level (The Brains): Inside Claude 4.6, text is processed through mathematical matrices called KV (Key-Value) Caches. Instead of re-reading text, the GPUs calculate the meaning of your system instructions once and build a "mathematical profile." When a cache point is triggered, the model freezes this calculated KV state inside the GPU memory. 2.The AWS Bedrock Level (The Manager): Normally, an LLM wipes its memory the millisecond an API call finishes. AWS Bedrock changes this. It takes your static prompt, creates a secure, unique cryptographic hash (fingerprint), and pins that KV memory block alive. When your next API request comes in, AWS Bedrock instantly hashes the new incoming prompt text. If the top section matches a saved fingerprint, Bedrock's router bypasses the standard pre-fill setup and routes your request directly to the GPU holding your frozen mathematical profile. It is exactly like loading a "Save Game" file instead of restarting a video game from Le

"You're spawning a fresh codex exec for every single call? Just run the app-server and reuse a thread. The prompt cache alone will pay for it." That sentence sounds so obviously right that nobody ever benchmarks it. We run a small bazaar of headless AI daemons — designers, reviewers, code workers — and each one shells out to OpenAI's Codex CLI dozens of times per work item. A resident codex app-server --stdio with warm threads looked like free money. We measured it four times. The prompt cache eventually hit 86% on the turns that mattered. The pipeline got slower than the boring baseline, and 39% more expensive in raw tokens. This is the story of why, and of what app-server is actually for. The setup was deliberately minimal: no resident daemon, no pool. The server is spawned privately for one formation , runs its turns, and is killed by a context manager on the way out — success or crash. That keeps the comparison honest: same lifecycle as a subprocess, and adopting it stays a measurement problem instead of an architecture debate. Trusting the probe Before writing any integration, we cloned the actual openai/codex source and pointed a probe at a real app-server: handshake, per-turn sandbox overrides, interrupts, forks, kill-and-resume. Eight checks, all green, plus three findings you won't find in any docs: cwd and sandboxPolicy are sticky when omitted — every turn must re-state them explicitly, or turn 3 silently inherits turn 2's write access. Token usage arrives on thread/tokenUsage/updated , not on turn/completed . An empty thread that never ran a turn is never materialized — you cannot resume it later. Pre-warming a thread pool is a trap. And the seductive one: on the probe's trivial two-turn thread, turn 2 reported 15,104 cached input tokens . See? The cache works. Ship it. That number cost us three more measurement runs to un-believe. The four runs Same fixture repo, same greenfield request, one real four-stage formation each: A: codex exec (baseline) B: app

Most ad-hoc timing code in Python looks like this: start = time . perf_counter () result = do_work () elapsed = time . perf_counter () - start stats [ name ]. append ( elapsed ) It works, until do_work() raises. Then the line that records the timing never runs, the exception propagates, and the one call that was probably slowest — the one that failed — is silently missing from your stats. If you're using timing data to find what's expensive, the failing case is exactly the one you can least afford to lose. timerx is a small, dependency-free Python timing library — a decorator, a context manager, and named stopwatches, all backed by one stats store. The one rule that shapes the whole implementation: a timing gets recorded whether or not the timed code raised. Decision 1: finally , everywhere, no exceptions to the rule @functools.wraps ( target ) def wrapper ( * args : Any , ** kwargs : Any ) -> Any : started = self . _clock () try : return target ( * args , ** kwargs ) finally : elapsed = self . _clock () - started with self . _lock : self . _record ( label , elapsed ) return wrapper The async wrapper is the identical shape with await added. The context manager ( _Lap ) does the same thing structurally, just split across __enter__ / __exit__ instead of try / finally : def __exit__ ( self , * exc_info : object ) -> bool : if self . _started is None : raise RuntimeError ( " timerx lap exited before it was entered " ) elapsed = self . _timer . _clock () - self . _started with self . _timer . _lock : self . _timer . _record ( self . _name , elapsed ) return False Note the return False — __exit__ deliberately never swallows the exception. It records the timing and lets the exception continue propagating unchanged, because a timing library has exactly one job here: observe, not intervene. A version that suppressed exceptions to "clean up" would be actively dangerous to drop into someone else's codebase. Three entry points — decorator, context manager, stopwatch — and all t