标签：#data

MVCC, WAL, vacuum, and replication slots under sustained delete load - and how to delete billions of rows without your database noticing Most "how to delete a lot of rows" articles stop at "batch it and delete children before parents." That advice is correct, it's table stakes, and everyone already knows it. This article is about everything after that - the parts that actually decide whether your cleanup runs quietly in the background for a week or pages you at 3 a.m. with a full disk and a replica that's six hours behind. The thesis: at scale, your DELETE statement is the easy part. The adversaries are the subsystems a delete feeds - MVCC tuple versioning, the write-ahead log, autovacuum, and the replication machinery. Bulk deletion is really an exercise in flow control across those subsystems . Get the SQL right and the systems wrong, and you'll still take production down. We'll assume PostgreSQL (the internals are PG-specific), a live OLTP primary with at least one physical replica and one or more logical/CDC consumers, and a target of hundreds of millions to billions of rows across many related tables. The one paragraph of "basics," so we can move on: delete in dependency order (referencing rows before referenced rows); collect parent keys once; never rely on ON DELETE CASCADE for huge deletes because you can't throttle a cascade. Done. Now the real material. 1. What a DELETE actually costs A delete is not "remove a row." Under MVCC it's "mark a row version dead and write that fact everywhere." For each deleted tuple, PostgreSQL: Sets xmax on the heap tuple to your transaction id. The row is still physically present; it becomes a dead tuple once your transaction commits and no snapshot can still see it. Writes a WAL record for the heap change. If this is the first modification of that page since the last checkpoint, it also writes a full-page image (FPI) - potentially 8 KB of WAL for a single-row change. Touches every index. Index entries aren't removed at delet

Six months ago I moved my RAG pipeline from Pinecone to self-hosted Qdrant. My vector search bill went from $210/month to $6.50/month. Same latency. Same recall. Here's exactly how. The Setup My app does document Q&A for legal contracts. The numbers: 5.2 million vectors (1536-dim, OpenAI embeddings) ~800K queries/month P99 latency requirement: < 50ms On Pinecone Serverless, this cost me roughly $210/month — storage plus read units plus write units for daily ingestion of new documents. What I Moved To A single Hetzner CX32 server: 4 vCPU, 8 GB RAM, 80 GB SSD €8.50/month (about $9.20) Qdrant running in Docker Automated daily backups to S3-compatible storage ($0.50/month) Total: ~$10/month. That's a 95% cost reduction. The Migration Was Easier Than Expected bash# Export from Pinecone (I used their scroll API) python export_pinecone.py --index legal-docs --output vectors.jsonl Start Qdrant docker run -d -p 6333:6333 -v ./storage:/qdrant/storage qdrant/qdrant Import python import_qdrant.py --input vectors.jsonl --collection legal-docs The whole migration took an afternoon. The Qdrant Python client is straightforward, and the API is surprisingly similar to Pinecone's. Performance Comparison I ran the same 10,000 test queries against both setups: MetricPinecone ServerlessQdrant Self-HostedP50 latency23ms4msP99 latency89ms12msRecall@100.970.97Monthly cost$210$10 The self-hosted Qdrant is actually faster because the data sits in memory on the same machine. Pinecone Serverless loads data from object storage on demand, which adds cold-start latency. When Self-Hosting Is a Bad Idea I want to be honest about the trade-offs: Don't self-host if: You have zero DevOps experience and no one on the team does You need 99.99% uptime SLA for enterprise customers Your vector count is growing unpredictably (10M one month, 100M the next) You're a team of 1-2 and every hour on infra is an hour not building product Do self-host if: Your scale is predictable (you know roughly how many vectors

Every backend engineer has seen this happen, you build an application on a relational database like MySQL, handling thousands of concurrent transactions effortlessly. Then, the business asks for a real time analytics dashboard. But when you run an aggregation query over historical data, suddenly the database that effortlessly managed live traffic starts thrashing, evicting your working set, and dragging application performance down. This isn't a tuning problem, a missing index, or a badly written query. It’s a fundamental architectural collision. OLTP (Online Transaction Processing) OLTP encompasses nearly every concurrent digital interaction triggered across a distributed system. A user downloading a PDF, a microservice firing an automatic maintenance log, a comment on a social feed these are all transactions. Data engineers rely on OLTP systems (like MySQL or PostgreSQL) to capture these concurrent streams of interactions for creating , updating and deleting records. The Tree Based In-Place Engine To reliably capture massive volumes of transactions without corrupting data or locking up the application, OLTP systems rely on a highly optimized, row oriented architecture built around the B+ Tree. Because they must provide immediate, atomic updates to existing records, transactional databases manage state through a strict sequence of physical tree traversal and in-memory page mutation: The B+ Tree Indexing: When a transaction reads or updates id: 1, the engine traverses a B+ Tree from the root, through the branch nodes, directly to the specific physical leaf node holding that row. This O(\log n) traversal guarantees a fast, isolated point-lookup. It ensures the application always hits the single version of the row without scanning irrelevant data. The Buffer Pool & In-Place Updates: OLTP systems perform in place updates. The database pulls the exact page containing id: 1 from the physical disk into memory (the Buffer Pool). The specific row is mutated directly in RAM

I Built a Consistent Hashing Ring in Pure Python and Finally Understood How Cassandra Distributes Data I've been using Cassandra and Redis Cluster for years. I knew consistent hashing was "how they work." But I never truly got it until I built one myself from scratch, in pure Python, with zero dependencies. This post is about what I learned doing that. The Problem Consistent Hashing Solves Imagine you have 3 servers and 1 million keys. The naive approach: server = hash(key) % 3 . It works great until you add or remove a server. Change 3 to 4, and almost every key remaps to a different server. In a caching layer, that means near 100% cache miss. In a database, it means massive data movement. That's the problem consistent hashing solves. When you add or remove a node, only a fraction of keys move. Specifically, 1/n of the keys, where n is the number of nodes. Building the Ring The core idea: place both nodes and keys on a circular number line from 0 to 2^32 (or any large integer). To find which node owns a key, walk clockwise until you hit a node. Here's the minimal version: import hashlib import bisect class ConsistentHashRing : def __init__ ( self , replicas = 150 ): self . replicas = replicas self . ring = {} # hash -> node name self . sorted_keys = [] # sorted hash positions def _hash ( self , key : str ) -> int : return int ( hashlib . md5 ( key . encode ()). hexdigest (), 16 ) def add_node ( self , node : str ): for i in range ( self . replicas ): virtual_key = f " { node } :vnode: { i } " h = self . _hash ( virtual_key ) self . ring [ h ] = node bisect . insort ( self . sorted_keys , h ) def remove_node ( self , node : str ): for i in range ( self . replicas ): virtual_key = f " { node } :vnode: { i } " h = self . _hash ( virtual_key ) del self . ring [ h ] idx = bisect . bisect_left ( self . sorted_keys , h ) self . sorted_keys . pop ( idx ) def get_node ( self , key : str ) -> str : if not self . ring : raise ValueError ( " Ring is empty " ) h = self . _hash

When I first started exploring Machine Learning, I made the same mistake most beginners do — I jumped straight into neural networks and model training without really understanding the Python underneath. I'd copy code from tutorials, get it running, and have zero idea why it worked. Then I started going through a structured Python-for-ML curriculum — and everything changed. This post is a distillation of that journey. If you're a CS student or early-career developer who wants to work seriously in ML/AI, here's the complete Python foundation you need — with the why , not just the what . Why Python Specifically? (It's Not Just Hype) Python isn't the fastest language. C++ blows it out of the water on speed — and I've personally used C++ for packet-capture modules in one of my ML projects. But Python dominates ML for one reason: the ecosystem . NumPy, Pandas, PyTorch, TensorFlow, Scikit-learn, Hugging Face — all Python-first. You don't choose Python for ML. The field chose it for you. Stage 1: Python Basics — The Foundation You Can't Skip Before you touch any ML library, you need these locked in. Variables and Data Types Python is dynamically typed, which feels nice at first but will bite you during data preprocessing if you're not careful. # These are all valid — Python infers the type name = " Parth " score = 8.97 is_enrolled = True year = 2025 For ML, the types that matter most are int , float , bool , and str — and knowing when Python silently converts between them (type coercion) can save you hours of debugging. Loops and Conditions — Your Data Iteration Backbone grades = [ 8.5 , 7.9 , 9.1 , 6.8 , 8.97 ] for g in grades : if g >= 8.5 : print ( f " Distinction: { g } " ) elif g >= 7.0 : print ( f " First Class: { g } " ) else : print ( f " Pass: { g } " ) Simple? Yes. But this exact pattern — iterate over a collection, branch on conditions — is the mental model for 80% of data cleaning code you'll write later. Functions and Lambda Expressions Functions are how you st

Traditional databases just can't keep up with high concurrency and low latency at the same time. The term "real-time" has become kind of meaningless. Everyone claims it, from batch-oriented cloud data warehouses to transactional database extensions. This makes picking the right architecture really hard without expensive trial and error. The best real-time analytics database in 2026 depends entirely on your workload shape. Key takeaways Real-time analytics (in this guide) = sub-second p95/p99 analytical queries on billions of rows, high concurrency , and milliseconds-to-seconds freshness . Best overall in 2026 for most workloads: ClickHouse (ingest throughput, query speed at scale, compression/TCO). Best for strictly predefined query paths via star-tree indexes: Apache Pinot . Best for time-series operational dashboards and observability: ClickHouse . ClickStack is its full observability offering for logs, metrics, and traces. Best for rigid ingestion-time roll-up aggregations: Apache Druid . Best for unified OLTP + real-time analytics: ClickHouse paired with its managed Postgres offering and native sync to ClickHouse , giving you a purpose-built OLTP engine and a purpose-built OLAP engine without rolling your own CDC pipeline. SingleStore is an alternative if you prefer a single HTAP engine for both. Traditional Data Warehouses: Snowflake and BigQuery are fine for batch BI if you already have one, but face latency, concurrency, and cost challenges under sub-second, high-concurrency workloads. Evaluate using 4 axes: ingest/freshness, latency under concurrency, TCO, operational complexity. What 'real-time analytics' means (and why warehouses and OLTP databases fail) Strict engineering thresholds define true real-time OLAP : sub-second query latency on complex aggregations, the ability to serve tens to thousands of concurrent queries per second (QPS), and data freshness measured in milliseconds to seconds. Traditional cloud data warehouses like Snowflake and BigQuery a

If you're building a modern data stack that requires either high-throughput transaction processing or large-scale analytical workloads, you've likely come across both Vertica and VoltDB (now rebranded as Volt Active Data). While both are distributed relational database management systems (RDBMS), they are architected for completely opposite use cases — choosing the wrong one can lead to 10x higher costs, missed latency SLAs, and poor application performance. In this guide, we break down every key difference between OpenText Vertica and Volt Active Data, with practical examples, real-world use cases, and best practices to help you make the right choice for your team. Table of Contents What is OpenText Vertica? What is Volt Active Data (Formerly VoltDB)? Core Differences Between Vertica and VoltDB Real-World Use Cases: When to Pick Which Best Practices & Common Mistakes Conclusion & Key Takeaways References What is OpenText Vertica? OpenText Vertica (formerly Micro Focus Vertica) is a columnar relational DBMS built exclusively for analytical (OLAP) workloads, first launched in 2005. As of 2026, the latest stable version is 26.1, with native lakehouse and Apache Iceberg export support for modern data ecosystems. Core Vertica Architecture Vertica's design is optimized for fast queries across massive datasets: Columnar storage : Data is stored by column instead of row, enabling significantly higher compression ratios and faster aggregation queries that only access a small subset of columns Massively Parallel Processing (MPP) : Query execution and data are distributed across hundreds of nodes for parallel processing Dual deployment modes : Enterprise Mode : Shared-nothing architecture with data stored locally on nodes for maximum performance Eon Mode : Compute and storage separated, using shared object storage (S3, GCS, ADLS) to scale compute independently of storage for cloud workloads Projections : Physical, sorted copies of data optimized for common query patterns (ins

PostgreSQL Error 22P01: Floating Point Exception PostgreSQL error code 22P01 is raised when a floating-point operation produces an exceptional result that cannot be represented as a valid number. This typically occurs during division by zero on float types, operations involving NaN (Not a Number), or arithmetic that yields Infinity . It is most commonly encountered in analytics, financial calculations, and data pipelines processing external or sensor data. Top 3 Causes 1. Division by Zero on Float Types Unlike integer division (which raises 22012 ), dividing a float by zero triggers a floating-point exception. This is especially common in ratio and rate calculations where the denominator can become zero at runtime. -- Problematic query SELECT total_sales :: float / total_orders :: float AS avg_order_value FROM daily_stats ; -- Safe fix using NULLIF SELECT date , total_sales :: float / NULLIF ( total_orders , 0 ):: float AS avg_order_value FROM daily_stats ; 2. NaN Values in Arithmetic Operations Data ingested from external systems, CSVs, or APIs may silently introduce NaN values into float columns. Once NaN participates in arithmetic, results become unpredictable and can trigger exceptions downstream. -- Detect NaN values (NaN is the only value not equal to itself) SELECT id , value FROM sensor_readings WHERE value != value ; -- Replace NaN with NULL safely UPDATE sensor_readings SET value = NULL WHERE value != value ; -- Filter NaN in aggregations SELECT device_id , AVG ( value ) FILTER ( WHERE value = value ) AS clean_avg FROM sensor_readings GROUP BY device_id ; 3. Infinity Arithmetic Conflicts Storing 'Infinity'::float or '-Infinity'::float is valid in PostgreSQL, but performing certain operations on them produces mathematically undefined results (e.g., Infinity - Infinity = NaN ), which can cascade into a floating-point exception. -- Check for Infinity values SELECT id , measurement FROM raw_data WHERE measurement IN ( 'Infinity' :: float , '-Infinity' :: float

Dulu, nyari query yang bikin database spike itu bisa makan berhari-hari. Yang nyari capek, yang nge-fix juga capek. Sekarang? Hitungan jam — dan bonusnya, sambil belajar hal baru juga. Ceritanya begini. Kalau kamu pernah kerja di aplikasi yang pakai ORM (Object-Relational Mapping — semacam "penerjemah otomatis" antara code dan database), pasti familiar sama situasi ini: database tiba-tiba lambat, kamu dapet raw query yang jadi biang kerok, tapi di codebase kamu nulis pakai syntax ORM yang bentuknya beda jauh dari SQL mentah itu. Buat yang belum pernah deal sama ORM, bayangin gini: kamu nulis pesan dalam bahasa Indonesia, lalu ada "penerjemah otomatis" yang convert jadi bahasa Jepang sebelum dikirim ke penerima. Suatu hari ada masalah di pesan yang terkirim — tapi kamu cuma bisa lihat versi bahasa Jepang-nya. Nyari bagian mana dari tulisan Indonesia kamu yang bikin terjemahan-nya bermasalah? Itu effort-nya yang bikin pengen balik tidur aja. Sekarang dengan bantuan Kiro, cukup kasih raw query + akses ke codebase, dia otomatis nyari bagian mana di code yang nge-generate query bermasalah itu. Yang dulu butuh berhari-hari, sekarang bisa selesai dalam hitungan jam — dan itu baru tahap investigasi, belum termasuk fixing-nya. Ceritanya Kenapa Bisa Pakai Kiro Akhir-akhir ini lagi aktif pakai Kiro di tempat kerja. Awal tahun lalu kantor dapat credits melalui program Kiro for Startup , jadi ya sekalian dimaksimalkan. Selain buat debug dan explore query di MS SQL Server, kadang pakai Kiro juga buat analisa log AWS CloudWatch — sambil kasih context aplikasi yang running biar analisa-nya lebih akurat dan gak generic. Di tulisan kali ini, saya mau sharing gimana pakai Kiro sebagai partner beberapa minggu terakhir buat improve query performance di aplikasi .NET Core. Kenapa "partner"? Karena Kiro-nya gak boleh langsung akses ke database — jadi wajib melalui perantara saya. Kita discuss, kolaborasi, dan nge-solve bareng. Bukan AI yang dikasih tombol terus disuruh jalan sendiri. Wakt

If you've ever tried building a sports betting application, odds tracker, arbitrage scanner, value betting tool, or sports analytics dashboard, you've probably experienced the same thing: You start with the exciting part. The idea. The algorithm. The UI. The business logic. And then reality hits. The Hidden Problem Nobody Talks About Most developers assume the hardest part of a betting-related project is the prediction model or arbitrage logic. In practice, the real challenge is data infrastructure. Before your project can calculate anything, you need: Live events Accurate odds Multiple bookmakers Consistent market structures Historical updates Reliable refresh rates And suddenly your "weekend project" turns into a full-time data engineering job. The Scraping Trap Most developers begin by scraping bookmaker websites. At first it seems simple: Open DevTools Find the API request Parse the response Save the data Done, right? Not quite. Within a few weeks you'll likely encounter: Changed endpoints Rate limits Cloudflare protection Different JSON formats Missing markets Broken parsers Increased maintenance costs Instead of improving your product, you're fixing scrapers. Again. And again. And again. Every Bookmaker Speaks a Different Language Let's say you want to compare odds from five sportsbooks. You quickly discover that every provider structures data differently. One bookmaker might return: { "home" : "Liverpool" , "away" : "Arsenal" } Another might return: { "team1" : "Liverpool" , "team2" : "Arsenal" } A third one could use: { "participants" : [ "Liverpool" , "Arsenal" ] } Now multiply that problem across: dozens of bookmakers hundreds of leagues thousands of events You end up spending more time normalizing data than building features. Real-Time Data Changes Everything Many projects work perfectly during testing. Then live data arrives. Odds can move multiple times within a minute. If your system refreshes too slowly: arbitrage opportunities disappear alerts become

One thing that confused me when I first started learning ClickHouse was the word FINAL . Because eventually you'll come across both: SELECT * FROM events FINAL ; and: OPTIMIZE TABLE events FINAL ; At first glance, they sound like they should do roughly the same thing. After all, both contain the word FINAL . But they actually solve two completely different problems. One affects query results. The other affects how data is physically stored. Understanding this distinction can save a lot of confusion when working with MergeTree tables. Why This Confusion Happens Most people encounter FINAL while working with engines like: ReplacingMergeTree SummingMergeTree AggregatingMergeTree Sooner or later they notice something like: SELECT * FROM users ; returns duplicate versions of rows. Then they discover: SELECT * FROM users FINAL ; and suddenly the results look correct. Naturally, many people assume: FINAL merges the table. But that's not exactly what is happening. What SELECT FINAL Actually Does When you run: SELECT * FROM users FINAL ; ClickHouse applies merge logic during query execution. Think of it as: "Show me what the table would look like if all relevant merges had already happened." The important part: It only affects the query result. After the query finishes: parts remain unchanged storage remains unchanged nothing is rewritten on disk The merge logic happens temporarily while the query is running. Once the query completes, the table is exactly as it was before. What OPTIMIZE FINAL Actually Does Now let's look at: OPTIMIZE TABLE users FINAL ; This is a completely different operation. Instead of modifying query results, ClickHouse physically merges parts on disk. The operation: rewrites data merges eligible parts removes obsolete versions creates larger merged parts Unlike SELECT FINAL , the effects remain after the command completes. This is a storage operation, not a query operation. The Simplest Way to Remember It Whenever I think about these commands, I use a v

TL;DR Traditional decoupled ETL pipelines (like the "Modern Data Stack") are too brittle and complex to handle the unpredictable, heavily nested data generated by AI and LLM features. Agentic data serving solves this by focusing on dynamic query routing and semantic discovery, letting AI agents discover and query data autonomously using schema-resilient tools and codified business logic. You can build an agentic data stack by pairing S3 storage with DuckDB's native JSON handling and schema-agnostic Parquet reading ( union_by_name=true ), eliminating failure-prone parsing steps. The open Model Context Protocol (MCP) replaces custom, hacky LangChain tools by providing a standard interface for agents to discover schemas and execute queries securely. The open Model Context Protocol (MCP) and DuckDB's embeddable architecture make it practical to connect agents directly to your data with minimal infrastructure overhead and elastic, consumption-based compute. For years, broken ETL jobs powered my pager and my morning coffee. I am a staff engineer, and like many of you, I have spent a ridiculous amount of my career babysitting data pipelines. It is a thankless job that often feels like patching holes in a sinking ship. You are not alone in this. A Forbes survey shows data teams notoriously spend up to 80% of their time just moving and cleaning data instead of doing the interesting work of analysis. And the financial magnitude of this bottleneck is staggering: the ETL market is projected to reach $20.1 billion by 2032 at a 13% CAGR. This proves that massive industry capital is flowing into solving these pipeline bottlenecks, but throwing more money at the same old architecture was not going to save my mornings. This constant firefighting was frustrating, but manageable. Then came the new mandate: build the data backbone for our next-gen AI and LLM-based product features. The unpredictability of the queries and the sheer complexity of the data, nested JSON everywhere, were th

TL;DR: This is not a cryptographic construction. It is a pragmatic engineering compromise for applications where encrypted storage is required but approximate alphabetical ordering is still useful. I sort encrypted strings using an external index: the sum of weighted Unicode code points for the first N characters with exponential positional weights, followed by quantization. Monotonicity is preserved, but accuracy predictably degrades after the first few characters. Not a cryptographic scheme; some ordering information leaks by design. The problem Some time ago, while implementing a project, I ran into the problem of sorting encrypted data in a database. I’d like to share the solution. I won’t go into detail describing the entire application. I’ll just say that, according to the required architecture, almost all data in the database must be stored exclusively in encrypted form: usernames, file names, tags, comments, dates, and so on (with the exception of identifiers and some system fields). That is, the table structure should be open, but the contents should not be. The encryption is symmetric: the same key is used for both encryption and decryption. This means that without the encryption key, even with a full database dump an attacker should not obtain any original data. And this is where two problems immediately arose: searching by the data without fully decrypting it, and sorting encrypted data. The first problem, with some caveats, is solved fairly simply. To search encrypted data, it is enough to additionally store hashes of the original values you plan to search on. This allows exact-match lookups (for example, users by login or files by tag) without storing the original values in plaintext. Yes, this won’t allow pattern searches, but it’s quite acceptable for the project’s goals. But sorting encrypted data turned out to be significantly more difficult. The solution A quick search showed that the problem is far from new, but there are no standard approaches t

Google recently announced that WebMCP is entering origin trials in Chrome 149. The new WebMCP standard proposal lets sites expose tools (e.g., JavaScript functions and HTML forms) to in-browser AI agents, which can thus reliably simulate user actions instead of resorting to possibly expensive (e.g., on-screen reading) and often unreliable guesswork (e.g., DOM scraping). By Bruno Couriol

DuckDB Data Inlining, SQLite Fossildelta OOB, Postgres 19 Temporal Data Today's Highlights Today's highlights include DuckDB's innovative data inlining for stream processing in data lakes, offering significant performance gains by eliminating the small files problem. Additionally, a critical out-of-bounds read vulnerability in SQLite's fossildelta extension and a peek into PostgreSQL 19's focus on temporal data capabilities are discussed. Data Inlining in DuckLake: Unlocking Streaming for Data Lakes (DuckDB Blog) Source: https://duckdb.org/2026/04/02/data-inlining-in-ducklake.html The DuckDB team has unveiled DuckLake’s new data inlining feature, designed to revolutionize how streaming data is managed in data lakes by effectively tackling the notorious “small files problem.” This issue, common in scenarios with frequent small updates or continuous ingestion, often leads to performance bottlenecks due to the overhead of managing numerous tiny files. DuckLake's solution involves intelligently storing these small updates directly within the catalog, thereby eliminating the need for physical small files on disk. This architectural innovation significantly improves the practicality of continuous streaming into data lakes, enabling more efficient real-time analytics. By inlining data, DuckDB reduces I/O operations and metadata management complexity, leading to substantial performance gains. A benchmark highlighted in the announcement demonstrates an impressive 926x speed improvement for certain operations, showcasing the feature's potential to transform data lake architectures for workloads requiring high-throughput ingestion and immediate query access without the traditional performance penalties. Comment: This DuckDB feature is a game-changer for data lake architectures, offering a simple yet powerful way to handle streaming data without the performance overhead of countless small files. Post: Out-of-bounds read in deltaGetInt() when input contains no in-buffer terminat

Winning fight against AI data centers gives people a "taste of political power."

Even moderately sized data centers can have an outsized local impact.

GBase 8a fully supports OLAP window functions, making it a powerful gbase database for analytical workloads. This guide uses real sales scenarios to demonstrate ROW_NUMBER / RANK , moving aggregates, LAG / LEAD for period‑over‑period comparisons, ROLLUP subtotals, and how these functions execute in an MPP environment. Window Function Syntax function_name () OVER ( [ PARTITION BY column ] -- window group [ ORDER BY column ] -- ordering within window [ ROWS | RANGE BETWEEN ... AND ...] -- frame definition ) Unlike GROUP BY , window functions do not collapse rows. Every row remains in the result set while still being able to “see” other rows in the window. Ranking Functions ROW_NUMBER / RANK / DENSE_RANK SELECT dept_id , salesperson , sale_amount , ROW_NUMBER () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS row_num , RANK () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS rnk , DENSE_RANK () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS dense_rnk FROM sales WHERE sale_date >= '2024-01-01' ; Sample output: dept_id salesperson sale_amount row_num rnk dense_rnk 101 Zhang San 95000 1 1 1 101 Li Si 95000 2 1 1 101 Wang Wu 82000 3 3 2 101 Zhao Liu 71000 4 4 3 Top 3 Sales per Department WITH ranked AS ( SELECT dept_id , salesperson , sale_amount , ROW_NUMBER () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS rn FROM sales WHERE YEAR ( sale_date ) = 2024 ) SELECT dept_id , salesperson , sale_amount FROM ranked WHERE rn <= 3 ORDER BY dept_id , rn ; NTILE: Bucketing SELECT dept_id , salesperson , sale_amount , NTILE ( 4 ) OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS quartile FROM sales WHERE YEAR ( sale_date ) = 2024 ; Cumulative and Moving Aggregates Year‑to‑Date SELECT dept_id , sale_month , monthly_amount , SUM ( monthly_amount ) OVER ( PARTITION BY dept_id ORDER BY sale_month ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS ytd_amount FROM ( SELECT dept_id , DATE_FORMAT ( sale_date , '%Y-%m' ) AS sale_month ,

I inherited a SQL Server database with 2.3 million rows. Queries took 45 seconds. Users were frustrated. Dashboards timed out. Here is exactly what I did. Step 1: Find the slowest queries I used SQL Server's query store to identify the top 10 worst performing queries. Step 2: Check the execution plan Missing index warnings everywhere. Also saw table scans on a 2 million row table. Step 3: Add targeted indexes Created two non-clustered indexes on the most filtered columns. No over-indexing. Just what the queries actually needed. Step 4: Rewrite the worst join One query was joining 6 tables with a cross apply that made no sense. Restructured to inner joins with proper filter ordering. The result 45 seconds down to 8 seconds. An 82% improvement. Real-time dashboards started working again. Key lesson Check what is actually slow before changing anything. Most people skip this and waste time optimizing the wrong thing. What is your fastest query optimization win?

I inherited a SQL Server database with 2.3 million rows. Queries took 45 seconds. Users were frustrated. Dashboards timed out. Here is exactly what I did. Step 1: Find the slowest queries I used SQL Server's query store to identify the top 10 worst performing queries. Step 2: Check the execution plan Missing index warnings everywhere. Also saw table scans on a 2 million row table. Step 3: Add targeted indexes Created two non-clustered indexes on the most filtered columns. No over-indexing. Just what the queries actually needed. Step 4: Rewrite the worst join One query was joining 6 tables with a cross apply that made no sense. Restructured to inner joins with proper filter ordering. The result 45 seconds down to 8 seconds. An 82% improvement. Real-time dashboards started working again. Key lesson Check what is actually slow before changing anything. Most people skip this and waste time optimizing the wrong thing. What is your fastest query optimization win?