标签：#Database

🚀 Overview — Why Semantic Search Matters Semantic search surpasses simple keyword matching because embeddings place texts in a high‑dimensional vector space where cosine similarity directly reflects intent. A dedicated vector store is therefore required to persist those embeddings and serve nearest‑neighbor queries efficiently. This post demonstrates a pinecone fastapi semantic search tutorial that wires a FastAPI service to Pinecone, showing the full data flow from embedding generation to similarity lookup. 📑 Table of Contents 🚀 Overview — Why Semantic Search Matters 🛠 Environment Setup — How to Install Dependencies 🐍 Python Virtual Environment 📦 Required Packages 📦 Building the FastAPI Service — How to Create the API 🧩 Data Model with Pydantic 🔗 Core FastAPI Application 🔎 Integrating Pinecone — How to Store and Query Vectors 🗂 Index Creation and Configuration 📤 Upserting Documents 🔎 Performing a Semantic Search 📊 Performance & Scaling — How Indexes Influence Latency 🟩 Final Thoughts ❓ Frequently Asked Questions How do I secure the Pinecone API key in production? Can I use a different embedding model? What happens if I need to change the index dimension? 📚 References & Further Reading 🛠 Environment Setup — How to Install Dependencies Creating a reproducible environment guarantees that the tutorial runs identically on any machine. 🐍 Python Virtual Environment $ python3 -m venv venv $ source venv/bin/activate (venv) $ python -V Python 3.11.5 Activating the virtual environment isolates package installations from the global interpreter. 📦 Required Packages $ pip install fastapi[all] uvicorn pinecone-client sentence-transformers Collecting fastapi[all] Downloading fastapi-0.109.0-py3-none-any.whl (48 kB) Collecting uvicorn Downloading uvicorn-0.24.0-py3-none-any.whl (66 kB) Collecting pinecone-client Downloading pinecone_client-2.2.2-py3-none-any.whl (81 kB) Collecting sentence-transformers Downloading sentence_transformers-2.2.2-py3-none-any.whl (1.1 MB) ... Successful

CAP Theorem Explained: Choosing Between Consistency, Availability, and Partition Tolerance in Databases Imagine you're trying to book a flight online, and just as you're about to pay, the website crashes. When you try to book again, you find that the flight is now sold out, even though the website initially showed available seats. This frustrating experience is a classic example of a database trade-off between consistency, availability, and partition tolerance. The CAP theorem, first introduced by Eric Brewer in 2000, states that it's impossible for a distributed data store to simultaneously guarantee more than two out of these three principles. In this post, we'll delve into the world of CAP theorem, exploring its fundamentals, real-world database examples, and design implications. Introduction to CAP Theorem Understanding the Basics of CAP Theorem The CAP theorem is based on three primary principles: Consistency : Every read operation will see the most recent write or an error. Availability : Every request receives a response, without guarantee that it contains the most recent version of the information. Partition Tolerance : The system continues to function and make progress even when network partitions (i.e., splits or failures) occur. Importance of CAP Theorem in Distributed Systems In distributed systems, where data is spread across multiple nodes, the CAP theorem plays a crucial role in understanding the trade-offs between these principles. By grasping the CAP theorem, developers can design more resilient and scalable databases that meet the specific needs of their applications. Brief Overview of the Blog Post This post will explore the CAP theorem in depth, using real-world database examples to illustrate the trade-offs between consistency, availability, and partition tolerance. We'll discuss the fundamentals of CAP theorem, examine CA, CP, and AP systems, and provide guidance on designing for each combination. By the end of this post, you'll have a solid un

Hello! I'm Satoshi Kaneyasu, DevOps engineer at Serverworks. In this article, I'll introduce the basic concepts and terminology of vector databases for those who are just starting to learn about them. Target Audience This article is aimed at beginners to vector databases. You may have heard that vector databases are related to LLMs and RAG, but aren't quite sure what they actually are. Think of this as written with that kind of reader in mind. What Is a Vector Database? A vector database is a database that stores data as vectors (arrays of numbers) and searches for data using "distance" or "similarity" between vectors. Traditional relational databases search for data using "exact match" or "partial match" (LIKE queries), but vector databases can search for things that are semantically similar . For example, searching for "weather in Tokyo" might return results like "temperature in Tokyo" or "weather conditions in Kanto" — data that differs as a string but is semantically related. Visualizing Vector Space In a vector database, all data is represented as points in a multidimensional space. When searching, the query is also converted into a vector, and data that is "close in distance" within that space is retrieved. This diagram represents it in two dimensions, but in a real vector database, proximity and distance are defined across many dimensions. Use Cases for Vector Databases Vector databases are used across a wide range of applications: Use Case Description RAG (Retrieval-Augmented Generation) Knowledge base search to provide external knowledge to LLMs. Allows internal documents and up-to-date information to be reflected in LLM responses Semantic Search Searching internal documents or FAQs by meaning rather than keywords. Handles spelling variations and synonyms Recommendation Recommending products and content whose vectors are close to a user's preference vector. Used as an alternative or complement to collaborative filtering Image Search Searching for similar im

The LedgerSync pipeline was inserting 1.5 million rows into PostgreSQL using pandas.to_sql() . It took four minutes per run. I switched to psycopg2's COPY command and it dropped to 18 seconds. Same data, same schema, same machine. That is not an optimization tip. It is the difference between a pipeline that fits in an Airflow schedule and one that does not. This article is about patterns like that: the ones that matter when you are building pipelines that run on a schedule, not when you are writing ad-hoc queries. Loading Data: to_sql vs execute_values vs COPY There are three ways to write rows from Python into PostgreSQL, and the performance gap between them is significant. pandas to_sql issues one INSERT statement per row by default, or a multi-row INSERT with method="multi" . It is the easiest to write and the slowest for any serious volume. psycopg2 execute_values batches many rows into a single multi-row INSERT VALUES statement. About 5x faster than to_sql for medium-sized loads. psycopg2 COPY streams rows directly to PostgreSQL using its native bulk-load protocol. No statement parsing, no row-by-row overhead. For LedgerSync at 1.5M rows, this was the one that mattered. import psycopg2 import io import pandas as pd conn = psycopg2 . connect ( " host=localhost dbname=proj_db user=proj_user password=proj_pass " ) def bulk_copy ( df : pd . DataFrame , table : str , columns : list [ str ]): buf = io . StringIO () df [ columns ]. to_csv ( buf , index = False , header = False ) buf . seek ( 0 ) with conn . cursor () as cur : cur . copy_from ( buf , table , sep = " , " , columns = columns ) conn . commit () print ( f " Loaded { len ( df ) } rows into { table } " ) Use COPY for initial loads and large backfills. For incremental daily writes of a few thousand rows, execute_values is fine and gives you more control over conflict handling: from psycopg2.extras import execute_values def bulk_insert ( rows : list [ dict ], table : str ): if not rows : return columns = list

Your API is fast. Your code is clean. Your architecture looks solid on paper. Then you hit 500,000 records and everything slows down. Queries that ran in 12ms now take 4 seconds. Your dashboards lag. Users start filing support tickets. Your on-call engineer is staring at a query plan at midnight wondering what went wrong. Nine times out of ten, the answer is indexing. Not missing indexes — wrong indexes. Indexes that exist but don't help. Indexes that actively hurt write performance without meaningfully improving reads. This is a breakdown of the most damaging database indexing mistakes in production SaaS systems — and how to fix them before they become incidents. Mistake 1: Indexing Everything "Just in Case" The most common mistake isn't under-indexing. It's over-indexing out of anxiety. New engineers especially fall into this pattern — add an index on every column that appears in a WHERE clause, just to be safe. Seems responsible. It isn't. Every index you add is a write tax. On every INSERT, UPDATE, and DELETE, PostgreSQL (or MySQL) has to update every index on that table. On a table with 8 indexes, every write touches 8 data structures. At low volume, this is invisible. At 10,000 writes per minute, it becomes your bottleneck. The fix: Audit your indexes regularly. In PostgreSQL: SELECT schemaname , tablename , indexname , idx_scan , idx_tup_read , idx_tup_fetch FROM pg_stat_user_indexes ORDER BY idx_scan ASC ; Any index with idx_scan = 0 or near zero hasn't been used since your last stats reset. That's a candidate for removal — not immediately, but after investigation. Mistake 2: Not Understanding Index Selectivity An index on a boolean column ( is_active , is_deleted ) is almost always useless. Here's why: selectivity measures how many distinct values exist relative to total rows. A boolean column has two values. If 95% of your rows have is_active = true , an index on that column tells the query planner almost nothing useful. It will often skip the index entire

AI applications are generating more data than ever before. From model telemetry and user interactions to observability events and real-time analytics, modern systems need infrastructure that can ingest, process, and query massive datasets with low latency. That's exactly the problem ClickHouse is targeting with its latest release. The update introduces improvements across query performance, memory management, Kafka integration, lakehouse support, and developer tooling. While many of these changes appear incremental on the surface, together they highlight a much larger shift happening across the industry. One of the most notable additions is improved memory management for large joins. ClickHouse can now automatically spill hash joins to disk when memory usage exceeds configured thresholds. Instead of failing due to memory pressure, queries can continue running using more efficient execution strategies. For teams working with large feature tables, event enrichment, AI telemetry, or observability data, this can significantly improve reliability. The release also expands ClickHouse's Kafka capabilities with Schema Registry integration, AvroConfluent write support, metadata mapping, and zone-aware communication. These improvements make it easier to integrate ClickHouse into real-time event pipelines while reducing latency and unnecessary cross-zone traffic in cloud environments. Another major focus is support for modern lakehouse architectures. Improvements for Apache Iceberg and Apache Paimon strengthen ClickHouse's ability to query data stored in open table formats while maintaining high analytical performance. As more organizations separate storage and compute, ClickHouse is increasingly positioning itself as a high-speed query layer on top of cloud-native data lakes. Performance optimization remains a major theme throughout the release. Improvements include faster JOIN execution, better ORDER BY LIMIT performance, enhanced JSON processing, smarter index pruning, redu

In this article, author Aaditya Chauhan discusses the limitations of RAG pipelines based purely on vector search and how an internal omni-search application using Reciprocal Rank Fusion (RRF) that combines BM25 and vector results, can enhance the search solution. By Aaditya Chauhan

PostgreSQL 19 Graph Queries & REPACK; SQLite Advanced SQL Patterns Today's Highlights PostgreSQL 19 introduces major features with SQL/PGQ for graph queries on relational data and a new REPACK command to combat table bloat. Meanwhile, the SQLite community explores advanced SQL patterns for complex 'first match' selection logic, pushing the boundaries of efficient embedded database querying. SQL/PGQ in PostgreSQL 19: Graph Queries Without the Graph Database (Planet PostgreSQL) Source: https://postgr.es/p/9kW PostgreSQL 19 is set to integrate native graph querying capabilities through the introduction of SQL/PGQ (Property Graph Queries) and the GRAPH_TABLE construct. This innovative feature allows users to perform sophisticated graph-like analytics, such as pathfinding and pattern matching, directly on their existing relational datasets without requiring data migration to a specialized graph database. The syntax employed for defining graph patterns closely resembles that of Cypher, making it intuitive for developers familiar with graph query languages. Developers will be able to define 'nodes' and 'edges' from their relational tables and then utilize the GRAPH_TABLE function within standard SQL queries to traverse and extract complex, interconnected information. This effectively transforms PostgreSQL into a multi-model database capable of handling both traditional relational workloads and demanding graph analytics side-by-side, within a single consistent environment. This significant enhancement expands PostgreSQL's utility across a wide range of applications, including social network analysis, fraud detection, and supply chain optimization, where understanding relationships between data points is critical. By embedding graph capabilities, PostgreSQL removes the need for separate graph database infrastructure, simplifying application architectures and leveraging the robustness and familiarity of the PostgreSQL ecosystem for new and existing projects. Comment: This is

During maintenance tasks like backup or inspection, you often need to switch a gbase database cluster to READONLY mode and back to NORMAL afterwards. This post shows how to perform the switch online with gcadmin — no downtime required. Tools and Prerequisites Tool : gcadmin , located on any Coordinator node. User : Must be the cluster installation user (default gbase ). Pre‑check : Ensure gcware services are running ( gcware_services status ). Scope : The mode change applies cluster‑wide; no need to repeat per node. Step‑by‑Step (READONLY → NORMAL) Step 1: Check the Current Mode # Cluster‑wide gcadmin showcluster # Specific VC in a multi‑VC environment gcadmin showcluster vc vc1 Look at the VIRTUAL CLUSTER MODE field. Proceed only if it shows READONLY . Step 2: Switch to Normal Read/Write Mode # Single VC or whole cluster gcadmin switchmode normal # Specific VC gcadmin switchmode normal vc vc1 The switch takes effect in seconds. The cluster synchronises the new mode to all nodes automatically — no process restart is needed. Step 3: Verify the Change Run gcadmin showcluster again. When VIRTUAL CLUSTER MODE shows NORMAL , the cluster is fully writable again. Production Notes Business impact : An online switch does not interrupt running read queries. Pending write operations queued during READONLY mode are executed automatically after the switch. Node to run on : Any single Coordinator node is enough. Privileges : Only the gbase user can switch modes. Reverse switch : To return to read‑only mode, use gcadmin switchmode readonly [vc vc_name] . Troubleshooting : If the command fails, check gcware service health and node status first, then retry. Companion Commands # Check all cluster processes gcluster_services all info gcware_services all info # View detailed VC information gcadmin showvc The mode‑switch mechanism in GBase 8a is built for high availability. Following the "verify → switch → re‑verify" workflow lets you change cluster modes safely and transparently in a g

Modern search engines do more than match keywords. When you search for "cozy mystery set in Paris" and get results for "atmospheric detective novel in France" that's vector search at work: documents and queries are converted into lists of numbers, called embeddings, and the search engine finds the documents whose numbers are closest to the query's. Manticore Search supports this natively. Under the hood, it uses a data structure called HNSW: a graph that connects nearby vectors, so it can find nearest neighbors quickly without scanning every document. That makes vector search fast enough to run on millions of documents in milliseconds. But HNSW has an inefficiency. Early in the traversal, almost every distance computation finds a better candidate than the ones already in the result set. As the search goes on, those improvements become rarer, but the algorithm keeps traversing the graph until it exhausts its exploration budget. By that point, the result set has often already converged, and the remaining work does little or nothing to improve it. Early termination fixes this by detecting that point and stopping early. The effect becomes more noticeable as k grows, where k is the number of nearest neighbors the query asks Manticore to return. Returning more neighbors requires more graph exploration, and much of that extra work happens after the result set has already stabilized. That also makes early termination more valuable, because it has more unnecessary work to cut. This gets more pronounced with vector quantization . Quantization compresses stored vectors to save memory, which slightly lowers search precision. To recover it, Manticore uses oversampling : it fetches 3x more candidates than requested, then rescores them using the original full-precision vectors. With the default 3x oversampling, HNSW explores many more candidates per query. Large k values often come from this kind of candidate expansion: an application may ask the vector index for hundreds or thous

0A000 feature not supported 는? PostgreSQL 에러 코드 0A000 은 현재 사용하려는 기능이 PostgreSQL에서 지원되지 않거나, 특정 컨텍스트에서는 사용할 수 없음을 의미합니다. 주로 트랜잭션 내부에서 허용되지 않는 명령을 실행하거나, 해당 버전의 PostgreSQL에서 아직 구현되지 않은 SQL 표준 문법을 사용할 때, 또는 복제(Replication) 환경의 제약으로 인해 발생합니다. 실무에서는 특히 CREATE DATABASE , VACUUM , CLUSTER 같은 명령을 트랜잭션 블록 안에서 실행하거나, Logical Replication 슬롯과 관련된 작업을 수행할 때 자주 마주치는 에러입니다. 주요 발생 원인 1. 트랜잭션 블록 내에서 허용되지 않는 DDL 명령 실행 PostgreSQL은 일부 DDL 명령을 트랜잭션 블록( BEGIN ... COMMIT ) 내에서 실행하는 것을 허용하지 않습니다. CREATE DATABASE , DROP DATABASE , CREATE TABLESPACE , DROP TABLESPACE , VACUUM , CLUSTER 등의 명령은 트랜잭션 컨텍스트 밖에서 단독으로 실행되어야 하며, 이를 무시하고 트랜잭션 내부에서 호출하면 0A000 에러가 발생합니다. 2. 특정 PostgreSQL 버전에서 지원하지 않는 문법 또는 기능 사용 SQL 표준에는 정의되어 있지만 PostgreSQL의 해당 버전에서 아직 구현되지 않은 기능을 사용할 때 이 에러가 발생합니다. 예를 들어, 구버전 PostgreSQL에서 LATERAL JOIN , MERGE 문, GENERATED ALWAYS AS (expression) STORED 컬럼 정의 등을 사용하거나, 특정 윈도우 함수 옵션 조합을 사용하는 경우 해당 에러를 만날 수 있습니다. 3. 논리 복제(Logical Replication) 또는 스트리밍 복제 환경에서의 제약 위반 Standby 서버나 Logical Replication 구독자(Subscriber) 측에서 쓰기 작업이나 특정 관리 명령을 실행하려 할 때 0A000 에러가 발생합니다. Hot Standby 상태의 서버에서 DDL을 실행하거나, Logical Replication 슬롯이 활성화된 상태에서 지원되지 않는 방식으로 복제 슬롯을 조작하려는 경우 이 에러를 마주치게 됩니다. 해결 방법 원인 1: 트랜잭션 블록 내 허용되지 않는 DDL 실행 트랜잭션 블록을 제거하고 해당 명령을 단독으로 실행하는 것이 핵심입니다. 아래는 잘못된 예제와 올바른 예제를 비교한 것입니다. ❌ 잘못된 예 (0A000 에러 발생) BEGIN ; CREATE DATABASE myapp_db WITH OWNER = myapp_user ENCODING = 'UTF8' LC_COLLATE = 'ko_KR.UTF-8' LC_CTYPE = 'ko_KR.UTF-8' ; COMMIT ; -- ERROR: 0A000: CREATE DATABASE cannot run inside a transaction block ✅ 올바른 예 (트랜잭션 블록 밖에서 실행) -- 트랜잭션 블록 없이 단독 실행 CREATE DATABASE myapp_db WITH OWNER = myapp_user ENCODING = 'UTF8' LC_COLLATE = 'ko_KR.UTF-8' LC_CTYPE = 'ko_KR.UTF-8' ; VACUUM 명령도 동일한 원칙이 적용됩니다. -- ❌ 잘못된 예 BEGIN ; VACUUM ANALYZE public . orders ; COMMIT ; -- ✅ 올바른 예: 트랜잭션 밖에서 단독 실행 VACUUM ANALYZE public . orders ; -- ✅ 특정 테이블만 선택적으로 VACUUM VACUUM ( VERBOSE , ANALYZE ) public . orders ; 애플리케이션 코드(예: Python psycopg2)에서 자동 커밋을 끈 상태로 VACUUM을 호출하는 경우도 흔한 실수입니다. import psycopg2 conn = psycopg2 . conn

Now that almost everyone has thought about or is actively integrating AI workflows into their projects, some might ask is this all worth the cost? Many think the current economics of the AI space don't scale and that there will be upward price movement. Others still might not be comfortable with sending their data to remote services for processing. Then there is the crowd that wants to deploy models in small spaces with limited compute. Are there ways we can deploy small models locally and run at a lower cost? Yes with Knowledge Distillation . Knowledge distillation can get a bad rap due to it's questionable use in training some Large Language Models (LLMs). But it's a perfectly valid way to transfer performance from a larger model to a smaller one. Especially when both models are yours and/or open. This article will explore progressive distillation which is a technique to incrementally transfer knowledge from a series of larger teacher models into a smaller student. Install dependencies Install txtai and all dependencies. pip install txtai [ pipeline - train ] datasets Setup the Training Pipeline The first step we need to do is setup up the training pipeline. We'll use the Hugging Face Training framework to build a series of models. The following code establishes a train method, test method and loads the classification training data. from datasets import load_dataset from transformers import AutoModelForSequenceClassification , AutoTokenizer from txtai.pipeline import HFTrainer , Labels def train ( teacher , student , distillation , ** kwargs ): trainer = HFTrainer () model = AutoModelForSequenceClassification . from_pretrained ( student , trust_remote_code = True ) tokenizer = AutoTokenizer . from_pretrained ( student , trust_remote_code = True ) return trainer ( ( model , tokenizer ), ds [ " train " ], columns = ( " sentence " , " label " ), maxlength = maxlength , teacher = teacher , distillation = distillation , ** kwargs ) def test ( model ): labels = Labels (

The Theorem That Changed How We Think About Databases In 2000, Eric Brewer stood at a conference and proposed a conjecture that would reshape distributed systems forever: "You can only guarantee two of these three properties at the same time: Consistency, Availability, and Partition Tolerance." Two years later, Seth Gilbert and Nancy Lynch proved it mathematically. It became known as the CAP Theorem — and every distributed system architect since has had to wrestle with it. It sounds abstract. But once you understand it, you'll never look at a database choice the same way again. You'll understand why Amazon DynamoDB and Google Spanner make opposite architectural choices. You'll know why your bank uses PostgreSQL while Twitter uses Cassandra. Let's break it down from first principles. The Three Properties C — Consistency Every read receives the most recent write, or an error. There's only one version of the truth — all nodes agree. Not the same consistency as ACID . CAP consistency (linearizability) means every read reflects the latest write across all nodes. ACID consistency means transactions don't violate database constraints. Different concepts, same confusing word. A — Availability Every request receives a non-error response — though it might not be the most recent data. The system is always up and answering. Note: "Available" in CAP doesn't mean "fast." It means "responds without error." A system that always returns a (possibly stale) answer is Available. P — Partition Tolerance The system continues operating even when network messages between nodes are lost or delayed. A partition is when part of your distributed system can't communicate with another part. The Unavoidable Truth: P Is Not Optional Here's the insight that makes CAP actually useful: In any real distributed system, partitions will happen. Networks fail. Cables get cut. Data centers lose connectivity. AWS regions go down. Since you must tolerate partitions (or have a single-server system, which does

AI applications are moving quickly from simple chatbots to systems that can search, reason, recommend, summarize, and act on live business data. For developers, that usually means wiring together databases, embedding models, vector search, rerankers, orchestration logic, and application code. For no-code AI builders, it often means waiting for those integrations to exist before an idea can become a working prototype. The MongoDB extensions for Dify help close that gap. With the new MongoDB Atlas and Voyage AI extensions, Dify builders can visually compose AI workflows and agents that connect directly to MongoDB data, perform semantic retrieval with Atlas Vector Search, improve result quality with Voyage AI embeddings and reranking, and optionally interact with operational documents through controlled database tools. The result is a practical path from idea to working AI application: less custom orchestration code, more reusable building blocks, and a smoother experience for both developers and no-code builders. Why Dify and MongoDB Belong Together Dify provides a visual environment for building AI apps, workflows, and agents. It makes it easy to connect user input, model calls, tools, prompts, and outputs into a working application. MongoDB Atlas provides the data foundation: flexible documents, operational queries, aggregation, full-text search, and vector search in one platform. Together, they create a powerful pattern: Dify orchestrates the AI experience — workflows, agents, prompts, tools, and user interactions. MongoDB Atlas stores and retrieves the data — documents, application records, knowledge sources, and vector embeddings. Voyage AI improves retrieval quality — embeddings for semantic search and reranking for precision. For a no-code builder, this means you can assemble a retrieval-augmented generation workflow visually. For a developer, it means the integration points are packaged as reusable Dify tools rather than one-off glue code. Meet the Extensions

The honest decision framework, not another flame war. The SQL vs NoSQL debate has been running for 15 years and it still generates more heat than light. Here's the framework that actually helps you decide. The Real Question It's not "SQL or NoSQL." It's: what does your access pattern look like? If your application is mostly reading and writing related data through well-defined queries — orders with line items, users with addresses, products with categories — relational databases are purpose-built for this. JOINs are not expensive when they're indexed. Transactions are not slow when they're scoped correctly. PostgreSQL handles 50 million rows comfortably on a single node. If your application is reading and writing self-contained documents with predictable access by a primary key, and you rarely need cross-document queries — user profiles, product catalogs, content management — a document database simplifies your code. No ORM mapping hell. No migration files for adding a field. If your application writes massive volumes and reads by partition key with eventual consistency — time-series data, IoT telemetry, activity feeds at scale — wide-column stores like Cassandra were built for this specific workload. The 2026 Reality PostgreSQL has eaten NoSQL's lunch in many areas. JSONB support means you can store and query unstructured data inside PostgreSQL with GIN indexes. You get the document model flexibility without giving up transactions, JOINs, and a 30-year ecosystem. For 80% of startups and mid-size companies, PostgreSQL is the only database you need. MongoDB has gotten more relational. Multi-document ACID transactions (since 4.0), schema validation, aggregation pipelines that look suspiciously like SQL. It's converging toward what PostgreSQL already does, but with a different starting point. DynamoDB dominates serverless. If you're in AWS and your access pattern is simple key-value with known query patterns, DynamoDB's pricing model (pay-per-request) and operational s

Inspired by a similar project called GenosDB and Cloudflare’s initiative to rebuild Next.js, I decided to rebuild GunDB with a modern coding style, incorporating improvements and addressing shortcomings in the original technology. I used the OpenCode tool with the Big Pickle model to rewrite the project in a new graph database called Garfo (the Portuguese word for “fork”), and I was impressed with the results and its practical applications. In this article, I’ll explain the technology and its improvements over GunDB. Introduction Garfo is a modern, browser-first fork of GUN.js — the decentralized, offline-first graph database. A fork of the original project that keeps the familiar GUN graph API while bringing meaningful improvements to the modern JavaScript ecosystem. Why a Fork? GUN.js is a revolutionary technology — a graph database that syncs in real time, works peer-to-peer, resolves conflicts automatically, and runs in the browser. However, the JavaScript ecosystem has evolved. TypeScript has become the standard, ES modules are the norm, and new transport layers like Nostr have emerged as promising decentralized protocols. Garfo was born to fill these gaps: a GUN rewritten with modern typing, designed with the browser as a first-class citizen, and with native support for the Nostr protocol. Key Features Familiar Graph API If you've used GUN before, you'll feel right at home. Garfo exposes the same chainable API: import Garfo from ' garfo ' ; const db = new Garfo ({ localStorage : true }); db . get ( ' users ' ). get ( ' alice ' ). put ({ name : ' Alice ' , status : ' online ' }); db . get ( ' users ' ). get ( ' alice ' ). on ( profile => { console . log ( ' Update: ' , profile ); }); All the classic methods are there: get() , put() , set() , on() , once() , map() . Optional Nostr Transport This is one of the most exciting additions. Garfo can use Nostr relays as a transport layer, allowing peers to exchange graph messages through public or private relays: const

Happy to announce HeliosProxy !! Far beyond a pooling tool, HeliosProxy ** is a next-gen programmable Postgres data-plane. **Works with PostgreSQL-compatible databases , not only HeliosDB. It starts as a PgBouncer-compatible wedge, then adds the operational surface teams usually build from multiple tools: connection pooling failover and transaction replay shadow execution anomaly detection edge cache controls admin REST API embedded admin UI signed WASM plugins OCI-style plugin artifacts Kubernetes operator Terraform and Pulumi providers 22 installable Claude/Codex operator skills Install operator skills: heliosdb-proxy install skills PostgreSQL #DevOps #SRE #Database #AIcoding

Event sourcing gets sold on immutability. You don't update, you don't delete, you only append, so the history is permanent. It mostly isn't. The events are immutable because your code agrees not to touch them, not because anything actually stops it. Underneath they're still rows in Postgres, and rows have a DBA with write access. A migration that "cleans up" old data. A 2 a.m. query run against the wrong connection. A backup restored with slightly different bytes in it. Change one of those rows and a replay won't blink. The aggregate rebuilds, the projections rebuild, everything looks fine. Usually the first person to notice is a customer whose balance is off, and by then the trail is cold. Chain each event into the next The trick is small. Give every row two extra columns: a hash of its contents, and the hash of the row before it. #1 AccountOpened prev=00000… hash=70be4f… │ ▼ #2 AmountDeposited prev=70be4f… hash=796018… │ ▼ #3 AmountWithdrawn prev=796018… hash=6a0260… The hash is SHA-256(previousHash || json(payload)) . Nothing exotic. The point is that each hash depends on the one before it. Edit a payload and its hash stops matching. Rewrite that hash to cover for the edit, and now the next row's pointer is wrong. You can't fix one without breaking the next. About forty lines of it Appending an event hashes it together with the previous one: public HashChainedEntry Append ( object payload ) { var previousHash = _entries . Count == 0 ? GenesisHash : _entries [^ 1 ]. Hash ; var hash = ComputeHash ( previousHash , payload ); var entry = new HashChainedEntry ( _entries . Count + 1 , payload , previousHash , hash ); _entries . Add ( entry ); return entry ; } internal static byte [] ComputeHash ( byte [] previousHash , object payload ) { var payloadJson = JsonSerializer . SerializeToUtf8Bytes ( payload , payload . GetType ()); var combined = new byte [ previousHash . Length + payloadJson . Length ]; Buffer . BlockCopy ( previousHash , 0 , combined , 0 , previousHash .

Si vous utilisez PostgreSQL, vous avez probablement déjà dû choisir entre une clé primaire BIGSERIAL et un UUID. Depuis des années, la version 4 (aléatoire) est le choix par défaut quand on veut un identifiant unique et distribué. Mais en 2026, une alternative plus récente s’impose : UUID v7, qui intègre un timestamp et promet de meilleures performances pour les index. Dans cet article, je vous explique concrètement ce qui change, avec des benchmarks PostgreSQL et des exemples de code, pour que vous puissiez décider en connaissance de cause. UUID v4 : le standard aléatoire et son problème d’index Un UUID v4 est constitué de 122 bits aléatoires. Cette absence totale de tri est sa force pour l’unicité, mais elle devient un handicap dans un index B‑tree, qui est la structure utilisée par PostgreSQL pour les clés primaires. Lorsque vous insérez un nouvel UUID v4, il a autant de chances de se retrouver au début de l’index qu’à la fin. Résultat : l’index se fragmente, les pages se remplissent mal, et les performances d’écriture se dégradent à mesure que la table grossit. J’ai reproduit un test simple sur PostgreSQL 16 avec 10 millions de lignes, en utilisant une table dont la seule différence est la colonne id : -- Table UUID v4 CREATE TABLE events_v4 ( id UUID DEFAULT gen_random_uuid () PRIMARY KEY , payload JSONB , created_at TIMESTAMPTZ DEFAULT now () ); -- Table UUID v7 (généré côté application, voir plus bas) CREATE TABLE events_v7 ( id UUID PRIMARY KEY , payload JSONB , created_at TIMESTAMPTZ DEFAULT now () ); Après insertion, voici les mesures : Type de clé Taille de l’index Fragmentation Latence moyenne d’insertion BIGINT ~214 Mo 0 % ~0,8 ms/ligne UUID v4 ~428 Mo (2×) 99 % ~4,8 ms/ligne UUID v7 ~428 Mo (2×) ~2 % ~1,1 ms/ligne Ce qui frappe, c’est la fragmentation quasi nulle de l’UUID v7. L’index reste compact et les insertions sont presque aussi rapides qu’avec un BIGSERIAL. L’UUID v4, lui, est plus de quatre fois plus lent à l’insertion sur ce volume. UUID v7 :

Thinking in terms of existence instead of lists SQL Pattern Series #1 of 21 A collection of practical SQL patterns that help developers recognize common solutions to recurring database problems. What You'll Learn In this article you'll learn: When EXISTS and IN solve the same problem The difference between set membership and existence Why the underlying mental model matters When I typically reach for EXISTS Most SQL developers write a query like this at some point: SELECT c . CustomerID , c . CustomerName FROM Customers c WHERE c . CustomerID IN ( SELECT o . CustomerID FROM Orders o ); And it works. But sometimes it isn't the best way to think about the problem. The Question Behind the Query Many SQL problems can be framed in two different ways. Set Membership Is this value in a set? WHERE CustomerID IN (...) Existence Does at least one matching row exist? WHERE EXISTS (...) Both approaches often return the same result. But they represent different mental models. The Presence Pattern The Presence Pattern is useful when you do not actually care about the values being returned from a related table. You only care whether a matching row exists. For example: Customers who have placed an order Users who have logged in Employees assigned to a project Products that have sales In these cases, the question is often: Does a related row exist? rather than: What values are contained in this list? Example Using EXISTS SELECT c . CustomerID , c . CustomerName FROM Customers c WHERE EXISTS ( SELECT 1 FROM Orders o WHERE o . CustomerID = c . CustomerID ); The subquery is correlated to the outer query. Conceptually, SQL asks: For this customer, does at least one matching order exist? As soon as the answer becomes true, the condition is satisfied. Why This Pattern Matters Many SQL developers initially learn syntax. Over time, they discover that query writing is really about choosing the right mental model. The Presence Pattern encourages you to think in terms of: existence relationshi