标签：#sql

DuckDB Data Inlining, SQLite Fossildelta OOB, Postgres 19 Temporal Data Today's Highlights Today's highlights include DuckDB's innovative data inlining for stream processing in data lakes, offering significant performance gains by eliminating the small files problem. Additionally, a critical out-of-bounds read vulnerability in SQLite's fossildelta extension and a peek into PostgreSQL 19's focus on temporal data capabilities are discussed. Data Inlining in DuckLake: Unlocking Streaming for Data Lakes (DuckDB Blog) Source: https://duckdb.org/2026/04/02/data-inlining-in-ducklake.html The DuckDB team has unveiled DuckLake’s new data inlining feature, designed to revolutionize how streaming data is managed in data lakes by effectively tackling the notorious “small files problem.” This issue, common in scenarios with frequent small updates or continuous ingestion, often leads to performance bottlenecks due to the overhead of managing numerous tiny files. DuckLake's solution involves intelligently storing these small updates directly within the catalog, thereby eliminating the need for physical small files on disk. This architectural innovation significantly improves the practicality of continuous streaming into data lakes, enabling more efficient real-time analytics. By inlining data, DuckDB reduces I/O operations and metadata management complexity, leading to substantial performance gains. A benchmark highlighted in the announcement demonstrates an impressive 926x speed improvement for certain operations, showcasing the feature's potential to transform data lake architectures for workloads requiring high-throughput ingestion and immediate query access without the traditional performance penalties. Comment: This DuckDB feature is a game-changer for data lake architectures, offering a simple yet powerful way to handle streaming data without the performance overhead of countless small files. Post: Out-of-bounds read in deltaGetInt() when input contains no in-buffer terminat

GBase 8a fully supports OLAP window functions, making it a powerful gbase database for analytical workloads. This guide uses real sales scenarios to demonstrate ROW_NUMBER / RANK , moving aggregates, LAG / LEAD for period‑over‑period comparisons, ROLLUP subtotals, and how these functions execute in an MPP environment. Window Function Syntax function_name () OVER ( [ PARTITION BY column ] -- window group [ ORDER BY column ] -- ordering within window [ ROWS | RANGE BETWEEN ... AND ...] -- frame definition ) Unlike GROUP BY , window functions do not collapse rows. Every row remains in the result set while still being able to “see” other rows in the window. Ranking Functions ROW_NUMBER / RANK / DENSE_RANK SELECT dept_id , salesperson , sale_amount , ROW_NUMBER () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS row_num , RANK () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS rnk , DENSE_RANK () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS dense_rnk FROM sales WHERE sale_date >= '2024-01-01' ; Sample output: dept_id salesperson sale_amount row_num rnk dense_rnk 101 Zhang San 95000 1 1 1 101 Li Si 95000 2 1 1 101 Wang Wu 82000 3 3 2 101 Zhao Liu 71000 4 4 3 Top 3 Sales per Department WITH ranked AS ( SELECT dept_id , salesperson , sale_amount , ROW_NUMBER () OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS rn FROM sales WHERE YEAR ( sale_date ) = 2024 ) SELECT dept_id , salesperson , sale_amount FROM ranked WHERE rn <= 3 ORDER BY dept_id , rn ; NTILE: Bucketing SELECT dept_id , salesperson , sale_amount , NTILE ( 4 ) OVER ( PARTITION BY dept_id ORDER BY sale_amount DESC ) AS quartile FROM sales WHERE YEAR ( sale_date ) = 2024 ; Cumulative and Moving Aggregates Year‑to‑Date SELECT dept_id , sale_month , monthly_amount , SUM ( monthly_amount ) OVER ( PARTITION BY dept_id ORDER BY sale_month ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS ytd_amount FROM ( SELECT dept_id , DATE_FORMAT ( sale_date , '%Y-%m' ) AS sale_month ,

I inherited a SQL Server database with 2.3 million rows. Queries took 45 seconds. Users were frustrated. Dashboards timed out. Here is exactly what I did. Step 1: Find the slowest queries I used SQL Server's query store to identify the top 10 worst performing queries. Step 2: Check the execution plan Missing index warnings everywhere. Also saw table scans on a 2 million row table. Step 3: Add targeted indexes Created two non-clustered indexes on the most filtered columns. No over-indexing. Just what the queries actually needed. Step 4: Rewrite the worst join One query was joining 6 tables with a cross apply that made no sense. Restructured to inner joins with proper filter ordering. The result 45 seconds down to 8 seconds. An 82% improvement. Real-time dashboards started working again. Key lesson Check what is actually slow before changing anything. Most people skip this and waste time optimizing the wrong thing. What is your fastest query optimization win?

I inherited a SQL Server database with 2.3 million rows. Queries took 45 seconds. Users were frustrated. Dashboards timed out. Here is exactly what I did. Step 1: Find the slowest queries I used SQL Server's query store to identify the top 10 worst performing queries. Step 2: Check the execution plan Missing index warnings everywhere. Also saw table scans on a 2 million row table. Step 3: Add targeted indexes Created two non-clustered indexes on the most filtered columns. No over-indexing. Just what the queries actually needed. Step 4: Rewrite the worst join One query was joining 6 tables with a cross apply that made no sense. Restructured to inner joins with proper filter ordering. The result 45 seconds down to 8 seconds. An 82% improvement. Real-time dashboards started working again. Key lesson Check what is actually slow before changing anything. Most people skip this and waste time optimizing the wrong thing. What is your fastest query optimization win?

Let's consider a table user . When we execute a query to find Emily , we are actually going through each record , looking whether the name column equals Emily . Indexes comes in when you want to speed this up. Let's create an Index with the name idx_users_name (the name can be anything, and it doesn't matter functionally): CREATE INDEX idx_users_name ON users ( name ); Now when you run SELECT * FROM users WHERE name = 'Emily' ; Postgres will use the index we just created (not by the name, the name is just for us) to execute that query, and the time complexity is reduced from O(n) to O(log n) .

Something is slow. Maybe a page takes forever to load, maybe a migration is hanging, maybe your Supabase dashboard just spins. You suspect a query is stuck somewhere in your database, but you can't see what's happening — Postgres doesn't exactly surface this on its own. Turns out it does. You just need to ask. Seeing what's running Postgres keeps track of every active connection and what it's doing in a system view called pg_stat_activity . You can query it like any table: SELECT pid , state , query , age ( clock_timestamp (), query_start ) AS duration FROM pg_stat_activity WHERE state != 'idle' ORDER BY duration DESC ; That gives you every non-idle process — its process ID, current state, the SQL it's running, and how long it's been at it. If something has been running for minutes when it should take milliseconds, you've found your problem. A few things worth knowing about the columns: pid — the process ID, which you'll need if you want to kill it state — usually active (running right now), idle in transaction (sitting inside an open transaction doing nothing), or idle (waiting for work) query — the actual SQL text query_start — when the current query began If you want to include the user and database to narrow things down: SELECT pid , usename , datname , state , query , age ( clock_timestamp (), query_start ) AS duration FROM pg_stat_activity WHERE state != 'idle' ORDER BY duration DESC ; The dangerous one — idle in transaction An active query that's been running for a while is usually just slow. An idle in transaction connection is a different kind of problem — it means someone (or some code) opened a transaction and never committed or rolled it back. The connection is doing nothing, but it's still holding locks, which can block other queries from running. These are the ones that tend to cause cascading slowdowns. If you see one that's been sitting there for longer than expected, it's almost certainly a bug in application code — a missing COMMIT , an unhandled e

PostgreSQL Error 2200G: Most Specific Type Mismatch PostgreSQL error code 2200G ( most_specific_type_mismatch ) is a SQL-standard data exception that occurs when a value's type does not match the most specific (most derived) type expected in a context involving type hierarchies, XML schema types, or user-defined structured types. It most commonly appears when working with composite types, domain hierarchies, or XML processing functions where type inheritance or derivation is in play. While relatively rare in everyday CRUD operations, it can be a significant pain point in enterprise applications with complex type systems. Top 3 Causes and Fixes 1. Composite or Domain Type Hierarchy Mismatch When a function expects a specific domain or composite type but receives a parent/base type, PostgreSQL raises 2200G. Always cast explicitly to the most specific required type. -- Define types CREATE TYPE base_info AS ( name TEXT , value INTEGER ); CREATE DOMAIN specific_info AS base_info ; -- Function expecting the specific domain type CREATE OR REPLACE FUNCTION handle_info ( data specific_info ) RETURNS TEXT AS $$ BEGIN RETURN ( data ). name || ': ' || ( data ). value ; END ; $$ LANGUAGE plpgsql ; -- WRONG: passing base type causes mismatch -- SELECT handle_info(ROW('test', 42)::base_info); -- CORRECT: explicit cast to the most specific type SELECT handle_info ( ROW ( 'test' , 42 ):: specific_info ); 2. XML Type Processing Mismatch Using XML functions like XMLTABLE or XMLCAST without explicitly matching the expected schema type can trigger this error. Always declare column types explicitly. -- Correct: explicitly typed columns in XMLTABLE SELECT * FROM XMLTABLE ( '//product' PASSING XMLPARSE ( DOCUMENT ' <products> <product> <id>1</id> <price>29.99</price> </product> </products> ' ) COLUMNS product_id INTEGER PATH 'id' , price NUMERIC PATH 'price' ); -- Explicit XMLCAST to resolve type ambiguity SELECT XMLCAST ( XMLQUERY ( '//price/text()' PASSING XMLPARSE ( DOCUMENT '<data><pri

SQLite ON CONFLICT DO SELECT Proposal, PostgreSQL 19 Features & SQLite Critical Bug Today's Highlights This week in databases, a proposal seeks to expand SQLite's ON CONFLICT clause to match PostgreSQL 19's DO SELECT for advanced conflict resolution. Concurrently, an early look at PostgreSQL 19 highlights key features improving performance and data management, while a critical out-of-bounds read bug in SQLite's fossildelta.c extension reminds us of the importance of low-level code security. Request: Support "ON CONFLICT DO SELECT" to match Postgres 19 (SQLite Forum) Source: https://sqlite.org/forum/info/81840ccfecf0885ba4418152d6c7f164de00d189b2cf7c682690151b0 This forum post proposes extending SQLite's ON CONFLICT clause to include a DO SELECT action, mirroring a feature anticipated in PostgreSQL 19. Currently, SQLite supports DO NOTHING and DO UPDATE for handling unique constraint violations. The proposed DO SELECT would allow an application to retrieve existing rows that caused the conflict, providing more granular control over conflict resolution beyond simply ignoring or updating the data. This feature would be particularly useful in complex data pipelines or replication scenarios where knowing which existing data caused a conflict is necessary for subsequent application logic, such as logging, merging, or initiating alternative processing paths. The discussion highlights the growing convergence of SQL features across different database systems and the desire for enhanced compatibility and expressiveness in SQLite. Implementing DO SELECT would empower developers to build more robust and intelligent conflict resolution strategies directly within their SQL statements, reducing the need for multi-step application-side logic involving separate SELECT queries after a failed INSERT or UPDATE attempt. Such an addition could streamline data ingestion processes and improve transactional integrity for embedded SQLite applications. Comment: This feature would be a game-ch

Stop writing SQL strings that look like a ransom note. Here's how to write queries that are readable, safe, and maintainable. The Problem With "Good Enough" SQL Formatting Most Python developers start here: user_id = 5 query = " SELECT * FROM users WHERE id = " + str ( user_id ) cursor . execute ( query ) It works. Until it doesn't — and when it breaks, it breaks badly : SQL injection, cryptic errors from mismatched types, and queries that take 45 minutes to debug at 2am. Let's fix that, permanently. 1. Never Concatenate User Input — Use Parameterized Queries This is rule #1 and it's non-negotiable. ❌ The Wrong Way (SQL Injection Waiting to Happen) username = request . args . get ( " username " ) # Could be: ' OR '1'='1 query = f " SELECT * FROM users WHERE username = ' { username } '" cursor . execute ( query ) If username is ' OR '1'='1 , your entire users table just got exposed. ✅ The Right Way: Parameterized Queries username = request . args . get ( " username " ) # psycopg2 (PostgreSQL) cursor . execute ( " SELECT * FROM users WHERE username = %s " , ( username ,)) # sqlite3 cursor . execute ( " SELECT * FROM users WHERE username = ? " , ( username ,)) # SQLAlchemy Core from sqlalchemy import text result = conn . execute ( text ( " SELECT * FROM users WHERE username = :name " ), { " name " : username }) The database driver handles escaping. You never touch it. This pattern is immune to SQL injection by design. Gotcha: Note the trailing comma in (username,) . Without it, Python treats the string as an iterable and passes each character as a separate parameter. This is one of the most common beginner bugs. # 💥 Bug: passes ('a', 'l', 'i', 'c', 'e') instead of ('alice',) cursor . execute ( " SELECT * FROM users WHERE username = %s " , ( username )) # ✅ Correct: single-element tuple cursor . execute ( " SELECT * FROM users WHERE username = %s " , ( username ,)) 2. Multi-Line Queries: Triple Quotes + Consistent Indentation For anything longer than one clause, use tri

PostgreSQL Error 2201X: invalid row count in result offset clause PostgreSQL error code 2201X ( invalid_row_count_in_result_offset_clause ) is thrown when the value provided to an OFFSET clause is not a valid non-negative integer. This commonly surfaces in applications that implement pagination using dynamic queries or user-supplied parameters, where the offset value may be negative, NULL, or a non-integer type. Top 3 Causes 1. Negative OFFSET Value The most frequent cause is a miscalculated page offset. When computing (page - 1) * page_size , a bad page number can produce a negative result. -- Triggers error 2201X SELECT * FROM orders ORDER BY created_at DESC OFFSET - 5 LIMIT 20 ; -- ERROR: invalid row count in result offset clause -- Fix: Use GREATEST to clamp the value to 0 SELECT * FROM orders ORDER BY created_at DESC OFFSET GREATEST ( 0 , - 5 ) LIMIT 20 ; 2. NULL Passed as OFFSET When an application fails to initialize or bind the offset parameter, NULL gets passed to the query. This often happens with ORMs or query builders where optional parameters are not explicitly set. -- Triggers error 2201X SELECT * FROM products ORDER BY id OFFSET NULL LIMIT 10 ; -- ERROR: invalid row count in result offset clause -- Fix: Use COALESCE to provide a default value SELECT * FROM products ORDER BY id OFFSET COALESCE ( NULL , 0 ) LIMIT 10 ; -- Parameterized query with safe fallback SELECT * FROM products ORDER BY id OFFSET COALESCE ( $ 1 :: BIGINT , 0 ) LIMIT COALESCE ( $ 2 :: BIGINT , 10 ); 3. Non-Integer Type (Float or String) Passing a float or a string that cannot be cleanly cast to a whole number causes this error. This typically happens when Python float values or unvalidated user input strings are interpolated directly into a query. -- Triggers error 2201X SELECT * FROM employees ORDER BY last_name OFFSET 10 . 7 LIMIT 5 ; -- ERROR: invalid row count in result offset clause -- Fix: Use FLOOR and explicit cast to BIGINT SELECT * FROM employees ORDER BY last_name OFFSET F

SQL is arguably the most widely used language in software engineering, yet it is often the least carefully written. Most teams enforce strict linting on their application code but leave SQL queries as a free-for-all. This guide covers the formatting rules that separate maintainable, team-friendly SQL from query spaghetti that haunts on-call rotations. Why Poorly Written SQL Is a Real Engineering Problem Unformatted SQL is not just an aesthetic issue - it is a correctness risk. Dense, run-on queries make it nearly impossible to spot accidental Cartesian products, missing GROUP BY clauses, or WHERE conditions that silently bypass indexes. By the time a performance problem surfaces in production, tracing it back to the root cause becomes a painful exercise in reading someone else's stream of consciousness. Rule 1: Keyword Capitalization SQL engines treat select and SELECT identically, but human readers do not. Always uppercase reserved keywords such as SELECT, FROM, WHERE, JOIN, GROUP BY, and ORDER BY. Keep table names, column names, and aliases lowercase. This single habit immediately creates a visual boundary between the logic structure of the query and the underlying data it operates on. Rule 2: Indentation and Clause Alignment Think of SQL clauses as layers in a data pipeline. Each major clause - SELECT, FROM, WHERE, GROUP BY, ORDER BY - should start at the left margin. Columns and filter conditions beneath them should be indented by 4 spaces (or 1 tab, as long as your team is consistent). This structure lets any reviewer skim the query top-to-bottom and understand the data flow at a glance. Rule 3: Trailing vs. Leading Commas This is a genuinely debated topic on data teams. Leading commas (placing the comma at the start of each new line) make version control diffs significantly cleaner when columns are added or removed. Trailing commas look more natural for developers coming from JavaScript or Python. Neither approach is wrong - what is wrong is mixing both styles

lakehouse has two storage areas ; Files and Tables Files Store structured, queryable data by sql Supports schema definitions and ACID transactions Tables Stores Raw or semi-structured data(CSV, parquet, JSON) No schema support Flexible for data explorations Schema allows for logical ordering of data on business functions or domain (sales,marketing etc) A dbo schema is enabled by default once a lakehouse is created Schema-enabled lakehouses also support schema-level permissions and cross-workspace queries using the four-part namespace Lakehouse mode : Lakehouse Explorer and SQL analytics endpoint Lakehouse Explorer: Allows managing, Update, create, upload of data.You can switch between tables in the lakehouse SQL anlytics endpoit : Does not allow modifying of the underlying data. You can query using TSQL at read only mode. Loading data into lakehouse: Upload data into files/ folders on the explorer Load into delta tables (no code) Transform using power query in dataflow gen2 INgest into notebooks using apache spark (programmatically) Use Copy data to move data into differnt sources using data factory pipelines -Shortcuts allow you to reference external data reducing copies. Access is managed by One Lake. Schema shortcuts map an entire schema to a folder of Delta tables in another lakehouse. SQL analytics endpoint provides read-only access to lakehouse tables using T-SQL queries. SQL USE CASES : adhoc queries, BI connections to power bi or azure data studio, Data validation You can use SQL views to store reusable query logic. Views are useful when you need to apply business rules, simplify complex joins, or provide curated data for downstream consumers. You can use Spark SQL for SQL-like queries or PySpark for programmatic data manipulation in Notebooks. Spark SQL works well for familiar SQL patterns. PySpark provides greater flexibility for complex transformations and integration with Python libraries. Power BI is the business intelligence and reporting layer in Fabr

If you're building any kind of personal-memory layer on top of SQLite — Claude Code conversation history, notes app, indexed knowledge base — there's a sharp edge in FTS5 that takes most people by surprise the first time they hit it. The default tokenizer ( unicode61 ) silently drops most Japanese substring queries. The fix is one line of SQL. But the failure mode is invisible enough that you can ship a personal search tool, use it for weeks, and never realize half your content is unreachable. This post walks through: The failure, reproducible in 20 lines of Python The one-line fix ( tokenize='trigram' ) and what it actually does under the hood A two-layer Git + SQLite design that uses this index in production for ~800 Claude Code conversations A separate FTS5 footgun around the - character that breaks time-blocking -style queries A free GitHub sample at the end if you want to run the same approach against your own data The failure, reproducible in 20 lines Spin up a fresh SQLite FTS5 table with the default settings and insert a single multilingual sentence: import sqlite3 conn = sqlite3 . connect ( " :memory: " ) conn . execute ( """ CREATE VIRTUAL TABLE notes USING fts5(content) """ ) conn . execute ( """ INSERT INTO notes(content) VALUES ( ' Tried time-blocking with the new 朝の運用フロー — ' ' the 9-11 slot worked but the 午後 part collapsed again. ' ) """ ) for q in [ " time " , " blocking " , " 朝の運用 " , " 午後 " ]: hits = conn . execute ( " SELECT count(*) FROM notes WHERE content MATCH ? " , ( q ,) ). fetchone ()[ 0 ] print ( f " { q !r} : { hits } hit(s) " ) Output: 'time': 1 hit(s) 'blocking': 1 hit(s) '朝の運用': 0 hit(s) '午後': 0 hit(s) Same row. Same content. English queries land, Japanese substring queries don't. That's not a bug, it's the default tokenizer behavior — and the default doesn't print a warning about it. The reason: unicode61 segments text on whitespace and unicode word-break properties. English words have spaces between them, so individual tokens are reco

DuckLake Spec, pg_background 2.0, and pgsql_tweaks 1.0.3 Enhance Database Ecosystem Today's Highlights This week's highlights include DuckDB's new DuckLake specification for simplified dataframe integration with data lakes, alongside key updates from the PostgreSQL community. We cover pg_background 2.0 for safer asynchronous SQL execution and the release of pgsql_tweaks 1.0.3 for enhanced monitoring and performance tuning. The DuckLake Spec Is so Simple, Even a Clanker Can Build One for Dataframes (DuckDB Blog) Source: https://duckdb.org/2026/05/04/ducklake-dataframe.html The DuckDB team has unveiled the DuckLake v1.0 specification, a significant step towards simplifying data lake interactions with dataframes. This specification aims to provide a robust yet straightforward framework for reading and writing dataframes directly from and to data lake storage, emphasizing ease of implementation. The announcement highlights the specification's simplicity, so much so that even AI can be leveraged to generate compatible dataframe reader/writer tools. This initiative promises to democratize data lake access, allowing developers and data engineers to integrate DuckDB's powerful analytical capabilities with their data lake architectures more seamlessly. By defining a clear standard, DuckLake facilitates the creation of a vibrant ecosystem of tools and connectors, enabling efficient data processing directly within the data lake context without complex ETL pipelines. This development positions DuckDB as an even more versatile tool for analytical workloads, bridging the gap between local data processing and large-scale data lake environments. The ability to easily build data lake connectors, potentially even with AI assistance, marks a notable shift towards more accessible and integrated data workflows. This could streamline operations for data scientists and analysts who frequently work with large datasets stored in various data lake formats, allowing them to leverage DuckDB's

When I started building OpinioZone , my goal was simple: create a platform where users could compare products, read reviews, and make informed buying decisions. At first, it seemed like a straightforward web application. Store products, display specifications, and allow users to browse information. However, as the platform grew, I quickly discovered that building a review and comparison website involves many technical and architectural challenges. Choosing the Technology Stack I selected ASP.NET Core as the primary framework because of its performance, flexibility, and long-term support. For data storage, I chose SQL Server since it provides strong reliability and works well with complex relationships between products, categories, reviews, ratings, and specifications. This combination allowed me to build a scalable foundation while keeping development manageable. Designing the Database One of the biggest challenges was designing a database structure that could support multiple product categories. A smartphone and a car have very different specifications, but the platform needed to handle both efficiently. Instead of creating completely separate systems, I designed a flexible structure that could store category-specific attributes while maintaining a consistent user experience. This decision made it easier to add new product categories without major database changes. Building Product Comparisons The comparison feature became one of the most important parts of the platform. Users expect side-by-side comparisons to load quickly and display meaningful differences between products. To achieve this, I had to optimize queries and carefully structure specification data. Performance became increasingly important as the number of products grew. SEO Challenges For a content-driven website, SEO is critical. Every product page requires: Unique titles Descriptions Structured content Internal linking Fast page loading One lesson I learned early was that technical SEO and content q

​We just deployed a major update to our SQL Server web sandbox, moving from an individual tool to a collaborative environment: ​✅ Group Rooms (Beta): Create private rooms using security tokens to chat and write code together in real time. ✅ Simulated Linked Servers (Beta): Connect with your friends' sandboxes to run cross-queries strictly in read-only mode to preserve performance. ✅ Gamification & XP: Earn experience points and unlock technical badges as you run queries or build database schemas. ✅ Social Layer: Manage your friends list and customize your public developer profile to showcase your achievements. ​Currently, all features are completely free to use during this phase of the platform. Try it out with your colleagues at tsqlsandbox.online and leave your feedback below!

PostgreSQL Error 2200D: invalid escape octet The 2200D: invalid escape octet error occurs in PostgreSQL when a bytea value contains an invalid escape sequence. This typically happens with the legacy escape format for binary data, where octet values must be represented as three-digit octal numbers in the range \000 to \377 . If the escape sequence falls outside this range or uses non-octal digits, PostgreSQL raises this error immediately. Top 3 Causes 1. Out-of-range octal values in bytea escape literals The bytea escape format only accepts octal values from \000 to \377 (decimal 0–255). Using values like \400 or non-octal digits like \9 will trigger this error. -- BAD: \400 exceeds valid octal range (max is \377) SELECT E ' \\ 400' :: bytea ; -- ERROR: invalid escape octet -- BAD: \9 is not a valid octal digit SELECT E ' \\ 9AB' :: bytea ; -- ERROR: invalid escape octet -- GOOD: valid octal escape sequences SELECT E ' \\ 377' :: bytea ; -- decimal 255 SELECT E ' \\ 101' :: bytea ; -- 'A' character SELECT E ' \\ 000' :: bytea ; -- null byte 2. Using escape format strings with hex output format Since PostgreSQL 9.0, the default bytea_output is hex . Applications that mix hex -format output back into escape -format input processing can generate malformed escape sequences. -- Check current bytea output format SHOW bytea_output ; -- GOOD: Use hex format (recommended for all new projects) SELECT ' \x DEADBEEF' :: bytea ; SELECT ' \x 48656C6C6F' :: bytea ; -- 'Hello' -- GOOD: Use encode/decode for safe conversions SELECT encode ( ' \x DEADBEEF' :: bytea , 'hex' ); -- output as hex string SELECT encode ( ' \x DEADBEEF' :: bytea , 'base64' ); -- output as base64 SELECT decode ( 'deadbeef' , 'hex' ); -- hex string → bytea SELECT decode ( 'SGVsbG8=' , 'base64' ); -- base64 → bytea 3. Incorrect escaping during data migration or manual SQL When migrating binary data from other databases (Oracle, MySQL) or writing raw INSERT statements manually, developers often confuse octal and

Last year, I examined RUM indexes within this series on multi-key indexing, demonstrating that they cannot substitute MongoDB's compound indexes for sorted queries. A year later, Microsoft has fixed this in the DocumentDB extension for PostgreSQL with an Extended RUM index that preserves the ordering of the keys, allowing an ordered scan rather than a bitmap scan. Let's revisit our pagination query to see how it performs now. I start a container with the latest DocumentDB (version v0.112-0 from May 26, 2026): docker run -d --name documentdb-local -p 10260:10260 -p 9712:9712 ghcr.io/documentdb/documentdb/documentdb-local:latest --username franck --password franck --start-pg I can connect to PostgreSQL on port 9712, where many extensions are installed, including the extended RUM index: docker exec -it documentdb-local psql -p 9712 postgres psql ( 17.10 ( Debian 17.10-1.pgdg13+1 )) Type "help" for help. postgres = # \dx List of installed extensions Name | Version | Schema | Description ------------------------- +---------+------------+------------------------------------------------------------ documentdb | 0.112-0 | public | API surface for DocumentDB for PostgreSQL documentdb_core | 0.112-0 | public | Core API surface for DocumentDB on PostgreSQL documentdb_extended_rum | 0.112-0 | public | DocumentDB Extended RUM index access method pg_cron | 1.6 | pg_catalog | Job scheduler for PostgreSQL plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language postgis | 3.6.3 | public | PostGIS geometry and geography spatial types and functions tsm_system_rows | 1.0 | public | TABLESAMPLE method which accepts number of rows as a limit vector | 0.8.2 | public | vector data type and ivfflat and hnsw access methods ( 8 rows ) postgres = # I can also connect to the MongoDB-compatible API: docker exec -it documentdb-local mongosh -u franck -p franck 'mongodb://localhost:10260/?tls=true&tlsAllowInvalidCertificates=true' Current Mongosh Log ID: 6a0b3b537d2a1c3471d1a7ba Connecting to: mo

For two years, our Sponsored Placements service booked limited ad inventory through Redis: a counter in Redis, a Redlock around the decrement, and a TTL key per hold. It oversold. Not catastrophically — consistently. 40–60 double-booked placements a month , each one a manual refund and an apology email to an advertiser. The root cause was never one bug. It was the architecture: two sources of truth that could not be made atomic with each other. The count lived in Redis; the ownership lived in SQL. No transaction spans both. The Redlock only ever protected the Redis half. The one mental shift SKIP LOCKED turns a contended table into a concurrent work queue. Instead of every request fighting over one counter, each request grabs different rows and ignores the ones someone else is holding. FOR UPDATE alone serializes — that's the experience that scares people off SQL locking. FOR UPDATE SKIP LOCKED is the opposite: a transaction that would have blocked instead skips the locked row and takes the next free one. One row per reservable unit, then: START TRANSACTION ; SELECT id FROM inventory_unit WHERE placement_id = 42 AND ( status = 'available' OR ( status = 'held' AND hold_expires_at < NOW ( 3 ))) -- self-healing expiry ORDER BY id LIMIT 2 FOR UPDATE SKIP LOCKED ; -- the whole trick UPDATE inventory_unit SET status = 'held' , reservation_id = 'uuid' , hold_expires_at = NOW ( 3 ) + INTERVAL 10 MINUTE WHERE id IN ( 1107 , 1108 ); INSERT INTO reservation (...) VALUES (...); COMMIT ; Two concurrent requests for the same pool lock different rows. Neither waits. The claim, the hold, and the reservation are one transaction — there is nothing to reconcile because there is nothing else. The numbers (8 weeks before vs 8 weeks after) Metric Redis + Redlock MySQL SKIP LOCKED Oversells / month 40–60 0 Reservation p95 210 ms 34 ms Reservation p99 540 ms 61 ms Throughput / instance ~600 RPS 1,400 RPS Lock-wait timeouts / day ~900 <5 Nightly reconciliation 9–14 min deleted Redis cluster

SQL tuning is the process of making a database query run faster and cheaper — cutting response time while minimizing the system resources it burns. Here's the playbook I actually use, from "this query is slow" to "this query is fixed," with the traps that bite people in the middle. The loop Tuning is iterative. The shape is always the same: Identify the problem. Find the slow query (logs, profiler, or user feedback) and measure a baseline — execution time and resource usage. You can't claim an improvement you didn't measure. Analyze & rewrite. Review the SQL for redundant joins, unnecessary work, and complex subqueries. Tighten the WHERE , select only the columns you need, convert subqueries to joins where it helps. Read the execution plan. Understand how the engine actually runs the query; find inefficient join orders and needless full scans. Revisit indexes. Evaluate whether existing indexes help; add or restructure as needed. Consider schema changes. If a column is updated so often that indexing it hurts, split it out. Sometimes the model is the bottleneck. Tune settings/hardware if it comes to that. Re-test and repeat. Apply changes, re-check the plan, confirm the gain, monitor. Reading an execution plan The execution plan shows how the DB will run your query — table scans, index access, join methods. Read it well and you can pinpoint where the time goes. Most engines expose it: EXPLAIN (MySQL/PostgreSQL), EXPLAIN PLAN FOR (Oracle), SET SHOWPLAN_ALL ON (SQL Server). Operators to know: Full Table Scan — reads every row. Happens when there's no suitable index, or the query can't use one. Index Scan — scans via an index; usually cheaper than a table scan. Index Seek — jumps to specific key values; very efficient, reads only the rows it needs. Nested Loops / Hash Join / Merge Join — the three ways to join two tables (more below). Sort — orders data; excessive sorting is a common performance drag. Three numbers that matter: Cost — estimated resources a step will cons