标签：#performance

I spent yesterday building purejq , a pure-Python implementation of jq. I expected it to be the slow-but-portable option. Then I benchmarked it against the jq package on PyPI (the C bindings everyone uses to run jq from Python) and got this, on a 100k-object array, in-process: workload purejq jq PyPI (C bindings) field-access stream 9 ms 368 ms filter + count 55 ms 442 ms map + aggregate 18 ms 444 ms group_by 112 ms 704 ms transform + sort 136 ms 899 ms Pure Python, 7-40x faster than the C extension. That number looked wrong to me too, so before publishing anything I made the benchmark script verify every output against the actual jq binary first ( tools/bench.py --verify ), re-ran everything as median-of-7, and gave the bindings their best-case API. The gap is real. Here's why. The serialization tax The C bindings wrap real jq, and real jq only speaks JSON. So every call does this: your dicts -> JSON text -> C parser -> jq evaluates -> JSON text -> dicts That round trip costs about 350-450 ms for 100k small objects on my machine, before any actual filtering happens. You can see it in the numbers: even a trivial field access pays the same ~400 ms floor as a group_by. purejq skips the trip entirely. It compiles the jq program once into Python closures and walks your dicts and lists directly: import purejq prog = purejq . compile ( " group_by(.team) | map({team: .[0].team, n: length}) " ) prog . first ( data ) # operates on your objects, no serialization The lesson generalizes beyond jq: when you embed a C library that has its own data model, the marshaling boundary is often more expensive than the work. An interpreter written in your language gets to skip the boundary, and that can buy back an order of magnitude. Surprise number two: the CLI beats the jq binary on big files This one I really didn't expect. End to end on a 93 MB file (1M objects), parse + filter + output: workload purejq CLI jq 1.8.1 binary single lookup 0.51 s 1.68 s filter + count 1.08 s 1.96 s grou

Earlier parts examined the performance characteristics of sequential and random access under single-threaded execution, and noted in passing the destructive effect of random access on the TLB. This part devotes full attention to the TLB: what it is, why a TLB miss is more severe than a cache miss, why a page table walk constitutes one of the longest dependency chains a CPU can encounter, how huge pages fundamentally alter TLB reach, and where memory-level parallelism falters in the face of TLB misses. Page Boundaries: Where the Prefetcher Halts Part III, in its discussion of prefetchers, noted a hard constraint: a prefetcher must not cross page boundaries on its own authority. The operating system manages virtual memory in units of pages (typically 4 KB, i.e., 64 cache lines). When a program reaches the end of one page and is about to step into the next, the prefetcher cannot proceed. The reason is that the next page may not reside in physical memory (it may have been swapped out to disk), or it may be an entirely invalid virtual address — if the prefetcher were to speculatively initiate an access to the next page, it would trigger a page fault: the OS would have to suspend the process and swap the page in from disk; in the case of an invalid address, the OS would terminate the process outright. From a security standpoint, the prefetcher neither can nor is permitted to autonomously cross page boundaries without TLB approval. Hence a performance brake appears every 4 KB — even when traversing an array sequentially, after every 64 cache line accesses the prefetch pipeline must pause and await confirmation of an address translation. This is not to say that modern CPU prefetchers are completely unable to cross pages. Intel's Next Page Prefetcher and AMD's equivalent mechanism can consult the TLB when approaching a page boundary — if the address mapping for the next page is already registered in the TLB, the prefetcher receives clearance to continue prefetching across th

🎬 This article is a companion to my YouTube video. Watch it here: Introduction In this video we are going to talk about Redis — what it is, what it does, and why it is an important part of my back-end stack. What is Redis? Redis is a free, open-source, in-memory data store. Unlike PostgreSQL which stores data on disk, Redis stores data entirely in memory — in RAM. This makes it extremely fast. Redis can handle millions of operations per second with sub-millisecond response times. Redis is most commonly used as a cache, a session store, a message broker, and a real-time data store. What is Caching? When your application queries a database, that query takes time — it reads from disk, processes the query, and returns the result. If the same query is made thousands of times per second, you are hitting the database thousands of times unnecessarily. Caching solves this by storing the result of a query in memory. The first request hits the database and the result is stored in Redis. Every subsequent request gets the result from Redis — which is in memory and therefore much faster — instead of hitting the database again. Think of it like a shortcut. Instead of driving the long route to the database every time, you take the shortcut through Redis. What Does Redis Do? Caching Store frequently accessed data in memory for fast retrieval. Database query results, API responses, computed values — anything that is expensive to compute and accessed frequently is a good candidate for caching. Session Storage Store user session data in Redis instead of the database. Since sessions are read on every request, having them in memory is significantly faster than a database lookup. Rate Limiting Track how many requests a user or IP address has made in a given time window. Redis's atomic increment operations make it perfect for implementing rate limiting. Message Queues and Pub/Sub Redis supports publish/subscribe messaging and message queues. Applications can publish messages to a channel a

Hey, welcome back. Last time we talked about missing parameterization in test scenarios. Today's mistake is similar in spirit. The test runs. The numbers look great. But what you've built isn't a load test. It's a hammer. ⚠️ The script works. The test is inhuman. Real users don't fire requests like a machine gun. They log in. They pause. They read. They click. They pause again. A typical user journey that takes 60 seconds in real life? Without think time, your script does it in just a few seconds. What this breaks Your throughput numbers are fiction. If users complete journeys 30x faster than reality, your RPS is inflated by 30x. You're not measuring capacity — you're measuring endurance under abuse. You stress the wrong things. Realistic concurrency surfaces real bottlenecks. A firehose of instant requests just overloads your connection pool and calls it a day. Production behaves nothing like your test. Because real users think. Your script didn't. 🛠 The fix Add randomized pauses between steps. Every major tool supports it: JMeter: Gaussian Random Timer, Uniform Random Timer etc. k6: sleep(Math.random() * 5 + 3) Gatling: pause(3.seconds, 8.seconds) Locust: time.sleep(random.uniform(3, 8)) 3–8 seconds between actions is a reasonable starting point. Check your analytics for what real sessions actually look like. Before your next run: Pauses between every major action? Randomized, not fixed? Does the timing feel human? If not — you're not testing load. You're testing collapse. Think time is one piece of the puzzle. But realistic load modeling goes deeper — it's about understanding how real users behave, how to translate that into a load profile, and how to design a test that actually reflects production. That's not something you patch with a timer. It's something you build from the ground up. If you want to understand the full system — from load model design to test execution to results that mean something — that's exactly what Performance Testing Fundamentals course

A few weeks after switching from Windows to Linux, I noticed something interesting. The hardware had not changed. The processor was the same. The RAM was the same. The SSD was the same. And yet, the laptop felt noticeably faster. Not necessarily because applications were completing tasks dramatically quicker, but because the entire system felt more responsive. Keyboard input felt immediate. Windows opened faster. Terminal commands appeared instantly. The desktop experience felt smoother. This raised a question: How can the same hardware feel different simply because the operating system changed? While I'm still learning, this is the mental model I've built so far. The Hardware Didn't Change Consider a laptop with: AMD Ryzen processor 16 GB DDR5 RAM NVMe SSD Modern integrated graphics When switching operating systems, none of these components change. The CPU does not suddenly become faster. The RAM does not magically increase. The SSD remains identical. From a hardware perspective: ```text id="u3m9xd" Before → Same Hardware After → Same Hardware So the difference must come from somewhere else. --- ## An Operating System Is Not Just a User Interface Many people think of an operating system primarily as the desktop they see. But an operating system does far more than display windows and icons. It manages: * Memory * CPU scheduling * Processes * Storage * Networking * Device drivers * Background services In other words: > The operating system decides how hardware resources are used. Two operating systems can therefore create very different experiences using the same hardware. --- ## Perceived Performance vs Raw Performance One thing I have learned is that performance is not always about benchmarks. A system can have excellent benchmark scores and still feel sluggish. Why? Because users experience responsiveness, not benchmark numbers. Examples include: * How quickly a window opens * How fast a menu appears * How responsive typing feels * How quickly applications launch

If you’ve ever spent hours debugging slow EC2 workloads or getting sticker shock from unexpected EBS IOPS charges, you’ve probably wondered if there’s a better storage option for temporary, high-performance data. AWS EC2 Instance Storage (also called Instance Store) is one of the most underutilized but powerful tools in the EC2 ecosystem—if you know how to use it correctly. This guide breaks down everything you need to know: core concepts, performance optimizations, use cases, limitations, and how it stacks up against EBS. By the end, you’ll be able to cut storage costs, boost workload performance, and avoid costly data loss mistakes. Table of Contents What Exactly Is AWS EC2 Instance Storage? Core Concepts of EC2 Instance Store Key Features That Make Instance Store Stand Out Which EC2 Instance Types Support Instance Store? Deep Dive: NVMe SSD Instance Store Volumes SSD Instance Store Performance Best Practices EC2 Instance Store vs EBS: Head-to-Head Comparison Top Real-World Use Cases for EC2 Instance Store Critical Limitations to Avoid Costly Mistakes Production-Grade Best Practices for Instance Store Root Volume Options: EBS-Backed vs Instance Store-Backed Instances EC2 Instance Store Pricing: No Hidden Costs Conclusion References What Exactly Is AWS EC2 Instance Storage? EC2 Instance Store is temporary block-level storage that is physically attached to the host server running your EC2 instance. Unlike standalone storage services like EBS, EFS, or S3, it is part of the EC2 service itself, with no network overhead between your instance and the storage disks. Its defining trait is its ephemeral nature: data stored on Instance Store only persists for the lifetime of the associated instance. If you stop, hibernate, or terminate your instance, all data on Instance Store volumes is permanently deleted. Core Concepts of EC2 Instance Store Before you start using Instance Store, make sure you understand these foundational rules: Device naming : Instance Store volumes are

Most merchants know they have unused files. Far fewer realize they're accumulating AI-generated media they never intended to keep. There's a problem quietly growing inside thousands of Shopify stores right now. It's not abandoned carts. It's not slow page speeds. It's not even the 400 unused product images you already know you should deal with. It's something newer, and most merchants have no idea it's happening. The Rise of AI-Generated Commerce Content Over the past two years, AI image tools have gone from novelty to routine. Shopify Magic. Canva AI. Midjourney. ChatGPT image generation. Adobe Firefly. Background removers. Lifestyle photo generators. Product shot enhancers. Merchants are using these tools constantly — to mock up new products, test background options, generate seasonal variants, create ad creatives, experiment with lifestyle photography. The workflow feels clean: generate a few options, pick the best one, move on. Here's what's actually happening on the backend. Every time you use Shopify's native AI tools to generate, edit, or enhance an image, Shopify quietly deposits files into your media library. Not just the one you kept. All of them. The rejected generations. The experimental edits. The "let me try one more variant" files. The abandoned attempts from six months ago when you were testing a new product that never launched. Every. Single. One. Most merchants assume the files they don't choose disappear. They don't. The lifecycle looks something like this: ┌─────────────────────┐ │ AI Image Generation │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ Rejected Variants │ │ • Drafts │ │ • Test Images │ │ • AI Edits │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ Hidden Media Files │ │ Accumulate Over Time│ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ AI Clutter │ │ Invisible Technical │ │ Debt │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ Reduced Media │ │ Governance │ │ • More Noise │ │ • Less Visibility │ │ • Hard

SQL tuning is the process of making a database query run faster and cheaper — cutting response time while minimizing the system resources it burns. Here's the playbook I actually use, from "this query is slow" to "this query is fixed," with the traps that bite people in the middle. The loop Tuning is iterative. The shape is always the same: Identify the problem. Find the slow query (logs, profiler, or user feedback) and measure a baseline — execution time and resource usage. You can't claim an improvement you didn't measure. Analyze & rewrite. Review the SQL for redundant joins, unnecessary work, and complex subqueries. Tighten the WHERE , select only the columns you need, convert subqueries to joins where it helps. Read the execution plan. Understand how the engine actually runs the query; find inefficient join orders and needless full scans. Revisit indexes. Evaluate whether existing indexes help; add or restructure as needed. Consider schema changes. If a column is updated so often that indexing it hurts, split it out. Sometimes the model is the bottleneck. Tune settings/hardware if it comes to that. Re-test and repeat. Apply changes, re-check the plan, confirm the gain, monitor. Reading an execution plan The execution plan shows how the DB will run your query — table scans, index access, join methods. Read it well and you can pinpoint where the time goes. Most engines expose it: EXPLAIN (MySQL/PostgreSQL), EXPLAIN PLAN FOR (Oracle), SET SHOWPLAN_ALL ON (SQL Server). Operators to know: Full Table Scan — reads every row. Happens when there's no suitable index, or the query can't use one. Index Scan — scans via an index; usually cheaper than a table scan. Index Seek — jumps to specific key values; very efficient, reads only the rows it needs. Nested Loops / Hash Join / Merge Join — the three ways to join two tables (more below). Sort — orders data; excessive sorting is a common performance drag. Three numbers that matter: Cost — estimated resources a step will cons

Published on : 2026-06-06 Reading time : 6 min Tags : #python #async #performance #optimization The Problem: Fake Async The supabase-async library claimed to be async but actually wrapped synchronous calls with ThreadPoolExecutor: # ❌ Fake async (old code) class SupabaseAsync : def __init__ ( self ): self . _executor = ThreadPoolExecutor ( max_workers = 3 ) async def select ( self , table : str ): loop = asyncio . get_event_loop () r = await loop . run_in_executor ( self . _executor , lambda : requests . get ( url ) # Sync call wrapped as async ) return r . json () Problems : Max 3 concurrent requests (not scalable) Thread overhead per request High memory usage No connection pooling Solution: httpx AsyncClient Use true async HTTP with httpx: # ✅ Real async (new code) import httpx class SupabaseAsync : def __init__ ( self ): self . _client : Optional [ httpx . AsyncClient ] = None async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 ) ) return self . _client async def select ( self , table : str ): client = await self . _get_client () r = await client . get ( f " { self . _base } / { table } " ) r . raise_for_status () return r . json () Performance Gains Metric ThreadPoolExecutor(3) httpx(10) Max concurrent 3 requests 10 requests Avg response 450ms 150ms Memory usage 250MB 180MB Throughput 6.7 req/s 20 req/s Real benchmark : 100 concurrent requests ThreadPoolExecutor: 15 seconds httpx AsyncClient: 5 seconds 3x faster ⚡ Migration Steps 1. Client Initialization with Lazy Loading async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 , max_keepalive_connections = 5 ) ) return self . _client 2. HTTP Methods (GET, POST, etc.) async def _request ( self , metho

Published on : 2026-06-06 Reading time : 6 min Tags : #python #async #performance #optimization 문제: 거짓 비동기 supabase-async 라이브러리는 이름은 async이지만, 실제로는 ThreadPoolExecutor로 동기 호출을 래핑하고 있었습니다. # ❌ 거짓 비동기 (기존 코드) class SupabaseAsync : def __init__ ( self ): self . _executor = ThreadPoolExecutor ( max_workers = 3 ) async def select ( self , table : str ): loop = asyncio . get_event_loop () r = await loop . run_in_executor ( self . _executor , lambda : requests . get ( url ) # 동기 호출을 async로 포장 ) return r . json () 문제점 : 최대 3개 동시 요청만 가능 (동시성 부족) 스레드 오버헤드 (각 요청마다 스레드 생성) 높은 메모리 사용량 해결책: httpx AsyncClient 진정한 비동기 HTTP 클라이언트인 httpx를 사용합니다. # ✅ 진정한 비동기 (수정된 코드) import httpx class SupabaseAsync : def __init__ ( self ): self . _client : Optional [ httpx . AsyncClient ] = None async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 ) ) return self . _client async def select ( self , table : str ): client = await self . _get_client () r = await client . get ( f " { self . _base } / { table } " ) r . raise_for_status () return r . json () 성능 개선 동시성 비교 지표 ThreadPoolExecutor(3) httpx(10) 최대 동시 요청 3개 10개 평균 응답 시간 450ms 150ms 메모리 사용량 250MB 180MB 초당 처리량 6.7 req/s 20 req/s 벤치마크 # 100개 동시 요청 처리 시간 ThreadPoolExecutor : 15 초 httpx AsyncClient : 5 초 → 3 배 빠름 마이그레이션 단계 1. 클라이언트 초기화 async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 , max_keepalive_connections = 5 ) ) return self . _client 2. 요청 메서드 async def _request ( self , method : str , url : str , ** kwargs ): client = await self . _get_client () if method == " GET " : return await client . get ( url , ** kwargs ) elif method == " POST " : return await client . post ( url , ** kwargs ) # ... 3. Context Manager 지원 async def clos

TL;DR — The DIFP protocol was designed to be data-compact and geo-aware from day one. We recently discovered it maps almost perfectly onto the Nostr event format. Here's how, and why it matters for decentralized food infrastructure. Background: What Is DIFP? DIFP (Djowda Interconnected Food Protocol) is an open protocol designed to sync food product data across distributed nodes — compactly, efficiently, and with geo-location awareness built in by default. One of its core design decisions is the PAD system (Preloaded Asset Distribution): Apps ship with a preloaded asset pack — item metadata, compressed images, category structure — all bundled at install time. Only price and availability need to travel over the wire during sync. This means the data footprint per product is tiny. Very tiny. Enter Nostr Nostr is a simple, open protocol for decentralized communication. One of its key specs: events support up to 64 KB of content . When we started exploring Nostr as a potential transport layer, we ran the numbers — and the fit was surprisingly clean. The Math: Products Per Event Baseline encoding A product represented with three fields: { "id" : 500 , "available" : true , "price" : 30000 } At this level of verbosity, a single 64 KB Nostr event can hold approximately: ~1,500 – 2,000 products Already useful. But we can do better. Optimized encoding Two key optimizations: 1. Drop the availability key — If a product entry exists in the JSON, it's available. If it's absent, it's not. No boolean needed. 2. Drop the field names — Instead of {"id": 500, "price": 30000} , just store: 500,30000 Field mapping is handled at the app level, not the protocol level. The device knows position 0 is the product ID, position 1 is the price (in smallest currency unit, e.g. cents). Result ~6,000 – 7,000 products per single Nostr event Possibly more, depending on the price distribution and ID ranges in a given catalog. Geo-Discovery: MinMax99 Cells DIFP uses a geo-cell system called MinMax99 to

Managing a multi-million product catalog on Magento presents unique challenges around performance, scalability, and operational efficiency. At Rave Digital, we recently undertook a Magento performance optimization project for a large-scale eCommerce merchant struggling with slow site speed, infrastructure bottlenecks, and backend instability. This use case breakdown details how we modernized their Magento architecture, optimized database performance, and scaled infrastructure to deliver a stable, high-speed shopping experience. This post is tailored for eCommerce managers, directors, and Magento merchants—especially those running Adobe Commerce or Magento Open Source platforms—who want to understand practical strategies for Magento architecture scaling and performance tuning for large catalogs. The Problem: Performance Bottlenecks in a Complex Magento Environment: Our client operated an enterprise Magento store with a multi-million product catalog. Despite Magento’s robust capabilities, the site suffered from: Slow page load times impacting user experience and SEO Scalability challenges as product volume and traffic grew Infrastructure bottlenecks causing backend instability and downtime Complex integrations and manual processes limiting operational efficiency Platform limitations in handling large catalog management and real-time inventory updates These issues collectively threatened the site’s ability to support growth and deliver a seamless customer experience. The client sought a comprehensive Magento platform modernization to address these challenges. Context: Why Magento Architecture and Infrastructure Matter Magento’s flexibility and extensibility make it ideal for enterprise eCommerce, but large catalogs require careful architecture and infrastructure planning. Key technical pain points include: Database performance under heavy read/write loads Indexing delays and cache invalidation impacting site speed Integration complexity with third-party systems and API

The speakers discuss the architectural challenges of executing safe data deletion across distributed datastores. Balancing durability, availability & correctness, they explain how to orchestrate multi-system deletion propagation without impacting live traffic. They share lessons on controlling tombstone accumulation, building continuous audit loops, and gaining trust with a centralized platform. By Vidhya Arvind, Shawn Liu

This is part thirteen in a series about managing the growing pile of skills, scripts, and context that AI coding agents depend on. Part ten covered the full improve pipeline — all five phases and how they connect. Part fourteen covers what 48 runs per day looks like in practice, including hardware benchmarks and the reliability bugs that surface at that frequency. The reflect pass inside akm improve has three execution modes. Most installs are still running the slowest one. Agent mode — the original — spawns an opencode or claude subprocess for each reflect call. The subprocess starts cold, acquires a session, assembles context, makes its LLM call, and exits. That cold-start overhead is real: each call takes approximately 30 seconds on a quiet machine. Run akm improve against a 69-ref stash and the reflect phase alone costs about 35 minutes. SDK mode eliminated the subprocess. The reflect call runs in-process, cutting per-call latency to 10–15 seconds. A 69-ref run drops to 12–17 minutes — better, but still bounded by round-trip overhead that the reflect task does not actually need. LLM mode removes the round trip entirely. The context for reflect is statically pre-assembled — no live tool calls, no file reads, no external context needed. A direct HTTP call to the LLM endpoint is sufficient, and it costs 6–10 seconds per call. A 69-ref run completes in 8–10 minutes. Mode Per-call latency 69-ref run agent (CLI subprocess) ~30s ~35 min sdk (in-process) ~10–15s ~12–17 min llm (direct HTTP) ~6–10s ~8–10 min The 3–4× end-to-end improvement is from eliminating overhead that was never necessary for what reflect does. Why Reflect Does Not Need an Agent The reflect pass takes a stash asset, examines its current content, and proposes a refined version. The inputs are fixed before the pass starts: the asset text, its metadata, and the improvement prompt. Nothing changes mid-call. No files need to be opened. No search queries need to fire. No external context needs to be pulled

There's a specific feeling you get after your third production caching incident. It's not panic....

Your API is fast. Your code is clean. Your architecture looks solid on paper. Then you hit 500,000 records and everything slows down. Queries that ran in 12ms now take 4 seconds. Your dashboards lag. Users start filing support tickets. Your on-call engineer is staring at a query plan at midnight wondering what went wrong. Nine times out of ten, the answer is indexing. Not missing indexes — wrong indexes. Indexes that exist but don't help. Indexes that actively hurt write performance without meaningfully improving reads. This is a breakdown of the most damaging database indexing mistakes in production SaaS systems — and how to fix them before they become incidents. Mistake 1: Indexing Everything "Just in Case" The most common mistake isn't under-indexing. It's over-indexing out of anxiety. New engineers especially fall into this pattern — add an index on every column that appears in a WHERE clause, just to be safe. Seems responsible. It isn't. Every index you add is a write tax. On every INSERT, UPDATE, and DELETE, PostgreSQL (or MySQL) has to update every index on that table. On a table with 8 indexes, every write touches 8 data structures. At low volume, this is invisible. At 10,000 writes per minute, it becomes your bottleneck. The fix: Audit your indexes regularly. In PostgreSQL: SELECT schemaname , tablename , indexname , idx_scan , idx_tup_read , idx_tup_fetch FROM pg_stat_user_indexes ORDER BY idx_scan ASC ; Any index with idx_scan = 0 or near zero hasn't been used since your last stats reset. That's a candidate for removal — not immediately, but after investigation. Mistake 2: Not Understanding Index Selectivity An index on a boolean column ( is_active , is_deleted ) is almost always useless. Here's why: selectivity measures how many distinct values exist relative to total rows. A boolean column has two values. If 95% of your rows have is_active = true , an index on that column tells the query planner almost nothing useful. It will often skip the index entire

This is a follow-up to SynaptoRoute: A Study in Local Semantic Routing . If you haven't read it, the short version is: SynaptoRoute is a zero-token semantic routing engine that classifies user queries into intents using local embeddings instead of LLM API calls. SynaptoRoute v0.3.0: Matching Semantic Router While Scaling to 50,000 Routes What Changed Since v0.2.0 When I published the first post, SynaptoRoute had just shipped dynamic batching and O(1) hot-reload. The throughput numbers were promising, but the accuracy story was incomplete. I had internal benchmarks but no comparison against a widely adopted baseline under identical, reproducible conditions. That gap is now closed. v0.3.0 is live on PyPI: pip install synaptoroute == 0.3.0 The Benchmarking Journey Getting to these numbers took multiple benchmark revisions. Early synthetic datasets produced catastrophic accuracy collapse and initially suggested that both SynaptoRoute and Semantic Router were performing poorly. After deeper investigation, the root cause turned out to be flaws in the dataset generation pipeline rather than limitations of the routing engines themselves. Several rounds of validation, failure analysis, threshold tuning, adversarial testing, and external benchmarking followed. All final results presented in this article come from independent public datasets with strict train/test separation, eliminating dataset leakage and benchmark inflation. That process was valuable because it forced the project to validate assumptions against real-world data instead of relying on synthetic benchmarks. The Benchmark That Actually Matters I evaluated SynaptoRoute against Semantic Router on two standard NLU datasets. Same embedding model ( BAAI/bge-small-en-v1.5 ). Same hardware. Same evaluation script. Same train/test splits loaded from HuggingFace. CLINC150 150 intents spanning 10 domains, plus an out-of-domain class. This is the standard stress test for intent routers. Metric SynaptoRoute Semantic Router

An LLM-written kernel benchmarked 38% faster on a microbench. Here is what kernel-level validation showed it actually did at runtime. TL;DR Multi-agent LLMs are now writing CUDA kernels (RightNow AI’s AutoKernel, Meta’s KernelEvolve, a multi-agent system claiming 38% speedup on Blackwell). Source-level benchmarks measure clean throughput on a single isolated kernel. They do not measure SM occupancy under co-scheduling, DRAM bandwidth saturation, dispatcher off-CPU during a real serving workload, or NCCL wait correlation with sibling kernels. Kernel-level validation closes that gap: an eBPF trace of the same kernel running under the same workload as production answers all four questions in one capture. The kernel-writing wave Three pieces of work in April surfaced the same pattern: agents generate CUDA kernels, then quote a single throughput number against a baseline. RightNow AI’s AutoKernel (announced Apr 6) – LLM agents iteratively rewrite CUDA kernels for a target metric, claiming substantial speedups on selected microbenchmarks. Meta’s KernelEvolve – similar shape: agents propose kernel variants, rank by throughput, keep the best. Multi-agent system on Blackwell (Apr 29 reports) – claims a 38% speedup on a public kernel benchmark using a coordinated agent setup. All three are real research, all three produce real kernels, and all three report numbers that come from microbenchmarks. The microbench setup is exactly what you want for the optimization loop. It is not what you get in production. What microbenchmarks do not see Run an LLM-generated kernel under nvprof or nsight-compute on an otherwise-idle GPU and the throughput number is real. Put the same kernel in front of a vLLM serving workload and four properties change immediately: SM occupancy under co-scheduling. The kernel that achieves 95% SM occupancy in isolation will achieve 40-50% with three other kernels sharing the same SMs. The optimizer never sees this regime. DRAM bandwidth saturation. A kernel tha

Modern search engines do more than match keywords. When you search for "cozy mystery set in Paris" and get results for "atmospheric detective novel in France" that's vector search at work: documents and queries are converted into lists of numbers, called embeddings, and the search engine finds the documents whose numbers are closest to the query's. Manticore Search supports this natively. Under the hood, it uses a data structure called HNSW: a graph that connects nearby vectors, so it can find nearest neighbors quickly without scanning every document. That makes vector search fast enough to run on millions of documents in milliseconds. But HNSW has an inefficiency. Early in the traversal, almost every distance computation finds a better candidate than the ones already in the result set. As the search goes on, those improvements become rarer, but the algorithm keeps traversing the graph until it exhausts its exploration budget. By that point, the result set has often already converged, and the remaining work does little or nothing to improve it. Early termination fixes this by detecting that point and stopping early. The effect becomes more noticeable as k grows, where k is the number of nearest neighbors the query asks Manticore to return. Returning more neighbors requires more graph exploration, and much of that extra work happens after the result set has already stabilized. That also makes early termination more valuable, because it has more unnecessary work to cut. This gets more pronounced with vector quantization . Quantization compresses stored vectors to save memory, which slightly lowers search precision. To recover it, Manticore uses oversampling : it fetches 3x more candidates than requested, then rescores them using the original full-precision vectors. With the default 3x oversampling, HNSW explores many more candidates per query. Large k values often come from this kind of candidate expansion: an application may ask the vector index for hundreds or thous

Overview PDF text extraction is a common pre-processing step in data pipelines — ingesting research papers, legal documents, or reports before embedding or indexing. Both pypdf and PdfPig are pure managed-code parsers: no native binaries, no OCR, no system PDF renderer. They implement the same PDF specification operations in their respective languages. This makes the benchmark unusually clean: the performance difference is entirely due to language execution speed, not library architecture differences. Benchmark Setup 200 recent arXiv PDFs (mixed technical papers, 5–40 pages each). Tested on subsets of 10, 50, 100, and 200 files. Both libraries extract all text from all pages; output is validated for page-count agreement and character-count agreement within 15% (pypdf and PdfPig decode whitespace and encoding tables slightly differently). Results PDFs Pages Python (pypdf) .NET (PdfPig) Speedup 10 ~120 ~0.9 s ~230 ms 3.9× 50 ~600 ~4.2 s ~810 ms 5.2× 100 ~1,200 ~8.5 s ~1.4 s 6.1× 200 ~2,400 ~17 s ~2.7 s 6.2× The speedup grows slightly with corpus size, suggesting pypdf has a per-document startup cost that compounds as PdfPig's JIT gets warmer. Why PdfPig Is Faster PDF parsing is byte-heavy: every page is a stream of PostScript-like operators (move, show text, set font, etc.). Each operator must be lexed, looked up in a dispatch table, and executed against a graphics state machine. In Python, each operator dispatch is a Python method call — the CPython bytecode interpreter has overhead per call regardless of what the method does. In .NET, the JIT compiles the dispatch loop to native code the first time it runs; subsequent pages pay only the cost of the actual work. Additionally, PdfPig's content-stream parser operates on ReadOnlySpan<byte> — zero-copy slicing through the raw page bytes with no intermediate string allocations. pypdf builds Python string objects for each token. Key Code // PdfPig — zero-copy span-based page extraction public Result Extract ( string path )