标签：#Postgres

Short answer: for semantic search over messy B2B catalog PDFs, I would spend the latency budget during ingestion, preserve page-level evidence, and keep the query path to one embedding plus one vector search; if a catalog must become searchable immediately after every upload, I would choose simpler deterministic chunks and defer enrichment. The decisive constraint is not the PDF parser or the language of the upload service. It is the quality-versus-latency boundary: descriptions often separate a product name, dimensions, compatibility notes, and exclusions across headings or pages, while a buyer expects one coherent result. A fast pipeline that loses those relationships produces plausible but unauditable answers. A sophisticated pipeline that blocks publication for too long fails a different operational requirement. For a Node.js RAG service, I treat the upload worker, embedding adapter, and Postgres repository as replaceable components. The durable contract is the evidence record. Each record needs a stable document version, a stable chunk identity, normalized text, page bounds, catalog identifiers, and the embedding configuration that produced its vector. That is the smallest design I trust for retries, reconciliation, and citations. How should Node.js RAG handle PDF upload chunking metadata and citations? The Node.js boundary should accept an upload, hash the original bytes, write an immutable document version, and enqueue ingestion under an idempotency key derived from the tenant, catalog, and file hash. Parsing and embedding can happen asynchronously. Search should read only a version whose ingestion status was committed as complete; otherwise a retry can expose half a catalog, which is especially awkward when two chunks describe the same SKU differently. Chunking comes after extraction, not during transport. Keep page boundaries from the parser, normalize repeated headers and whitespace without rewriting the source, then group adjacent blocks around product st

You've seen the headlines by now. An agent in Cursor wiped a company's production database, backups and all, in about nine seconds. Replit's agent nuked another company's prod. Same shape every time: the agent was sure of itself, the SQL was valid, and nobody was in the loop to say wait. Those are the loud failures. The fix for them is boring and you already know it. Don't hand an agent write access to prod. Read-only by default, propose instead of apply, keep a human on the button. But there's a quieter version that a permissions policy won't catch, and that's the one I want to talk about. Your agent is probably doing it right now. It looks completely fine in the diff. The migration that passes review and still takes the site down Ask an agent to make an email column unique. It writes: ALTER TABLE users ADD CONSTRAINT users_email_unique UNIQUE ( email ); Correct SQL. Does exactly what you asked. It sails through review because there's nothing to see. Then on a users table with any real size, it grabs an ACCESS EXCLUSIVE lock and scans every row to build the unique index, and for the whole length of that scan nothing else can read or write the table. The API starts timing out. The connection pool fills. Now you're in an incident over a one-line migration that everybody approved. The agent didn't do anything a decent junior engineer wouldn't have done. That's the trap. The danger isn't the SQL, it's the lock the SQL takes, and you can't see a lock by reading a statement. You'd have to know Postgres locking cold: which DDL grabs which lock, and for how long, and what it shuts out while it holds. And you'll still miss one at 2am. I got tired of missing them. So I measured one. What the lock actually costs I ran the same schema change two ways against a real Postgres 18. Fifty million rows, twenty connections doing ordinary traffic. The unsafe version was a plain SET NOT NULL , which also scans under ACCESS EXCLUSIVE . The safe version was the NOT VALID then VALIDATE da

If you built a Supabase app quickly - with an AI coding tool or by hand - the row-level-security policies were often written last, or generated for you. That is fine. What is not fine is shipping without knowing whether those policies actually isolate one user's rows from another. Supabase ships a database linter, and you should run it first - it is free and it catches the obvious cases: RLS switched off, and RLS switched on with no policy behind it. But a linter checks whether a policy exists , not whether the policy is correct . Those are different questions, and the second one is where cross-user leaks live. The 2-second test I put a minimal, synthetic reproduction on GitHub: supabase-rls-leak-demo . Same test suite on two branches, differing only by db/policies.sql : broken -> 4 failed, 1 passed (an authenticated user reads another user's row) fixed -> 5 passed npm ci npm run test :ci No Docker, no Supabase project, no credentials. The tests run PostgreSQL in PGlite locally and exercise database-level row security. They do not model Supabase Auth, PostgREST, the Data API, or the network path - the result proves only the row-level gate in the fixture, which is exactly the gate people get wrong. On broken , the failing assertion is readable on purpose: x does not let user B read any row owned by user A -> user B received 1 row(s) belonging to another user: ["A: card ending 4471, expiry 09/29"] (That is synthetic seed data, not a real card.) Run the free checks against your own database The repo also ships audit/rls-audit.sql - nine read-only queries against the system catalogs, MIT-licensed, nothing to install and nothing to send anywhere. Every one is SELECT -only, so it is safe to paste into the Supabase SQL editor. They tell you: RLS coverage per table Every policy and the roles it actually applies to (an empty roles array means no TO clause, so the policy is evaluated for anon too) The effective write check, and which columns its predicate never mentions What

ReBAC (relationship-based access control) decides access based on how entities are connected to each other, rather than on a role attached to the user. Nowhere is it written that you can see that repository. You see it because a chain of relationships leads you there. It's a model I like, and I want to say that up front, because what follows isn't a criticism of ReBAC. I spent a few weeks integrating OpenFGA into a prototype to use it properly: declarative model, sixteen test scenarios, a hundred and twenty assertions running offline in two seconds with no database and no application. It worked well. I removed it anyway. Not because of a bug, and not because of check latency. I removed it because none of the tools I tried gives me a usable answer to the second question every application asks. The check is fast. The list isn't. "Can this user see this object?" resolves in milliseconds. The problem is that the first screen after login is almost always a list. And "which objects can this user see?" looks like the same question reversed, but it isn't: nowhere is it recorded which objects are reachable, which is the point of ReBAC seen from the other side. In the first case you hold two things and walk the graph from one to the other. In the second you hold only the user, and the set of possible answers is everything that exists in the system. Three routes, and where each one stops Filter afterwards. Normal query with its normal pagination, then you send the twenty-five ids to the service and drop the ones that don't pass. The result is correct, but the total at the bottom of the page is the one from before the filter, so it's a lie. And the user with access to a small slice gets three rows out of twenty-five. Filter first. You ask the service which objects the user holds that permission on and hand them to the database. Except the list arrives whole. There's no real pagination to draw twenty-five from. It ends up as an IN with thousands of identifiers. A local index. Yo

You can load-test search latency with a script and a graph. Relevance has no such gauge by default, so most teams ship a new ranking rule, eyeball a handful of queries, and hope nothing important regressed. The fix is a small, boring relevance test suite: a fixed set of queries, human-judged expected results, and a metric you compute the same way every time — so "did this ranking change help?" becomes a number you can diff, not an argument you have in Slack. This post is a build guide. By the end you'll have a judgments file, a scorer that outputs precision@k, MRR, and nDCG, and a before/after comparison you can wire into CI. The examples use Postgres full-text search, but the harness is engine-agnostic — Elasticsearch, Meilisearch, or a vector store all slot into the same shape. Why can't I just load-test relevance the way I load-test latency? Latency is a property of the system. Relevance is a property of the match between a query and what a human expected to see — and that judgment lives outside the database. A commenter on an earlier post about running Postgres search in production put it well: latency can be load-tested, but quality needs query sets, expected result buckets, bad-query examples, and a way to compare changes before shipping a new ranking rule. That's the whole job, and none of it comes for free with your index. The trap is thinking a passing query proves relevance. SELECT ... WHERE tsv @@ query returning rows tells you the index matched. It says nothing about whether the right rows landed in the top 5, which is all a user ever sees. The takeaway: relevance is measured against human judgments, not row counts — so the first artifact you build is the judgments, not the query. Building the golden query set Start with 20–50 real queries. Pull them from your search logs if you have them (the head terms plus a long tail of specific ones), or write them from real user intents if you don't. For each query, mark which documents should come back and how rel

Two customers are looking at the same product. One unit left. Within the same second, both click Pay. If your checkout reads the stock count, decides there's enough, and then writes the decrement, both requests pass the check and both succeed. You've now sold two units of something you had one of. That's overselling, and it's not a rare edge case — it's the default behaviour of any checkout that treats "check stock" and "reduce stock" as two separate steps. The window is small, but on a product that's nearly sold out, or during a launch when everyone hits the same SKU at once, small windows fire constantly. I've built the order pipeline for two production e-commerce platforms — pikkuna.fi and pi-pi.ee — where concurrent webhooks and concurrent checkouts hit the same order and product rows. This is the layer I reach for when a store sells finite stock. I covered the bare SELECT ... FOR UPDATE primitive briefly in PostgreSQL Production Patterns ; this article is the whole system built on top of it — reservations, multi-line carts, the payment window, and the parts that actually bite you in production. When You Don't Need Any of This Start with the honest disclaimer, because it decides everything downstream. Both pikkuna.fi and pi-pi.ee are made-to-order . A vinyl curtain is cut to the customer's dimensions; a waterless urinal system ships from a supply chain, not a shelf with a hard unit count. When there's no fixed quantity to run out of, overselling isn't a failure mode — you can't sell the tenth unit of something you manufacture on demand. So neither of those platforms needs a row lock on a stock column, and I didn't build one there. You need this article when you sell discrete, finite stock : limited runs, event tickets, one-off items, anything where "5 left" is a real number and selling the sixth is a promise you can't keep. If your catalogue is print-on-demand, made-to-order, or backed by effectively unlimited supply, stop here — the locking below is complexity

This is Part 2 of the Directus + Coolify series. If you're new here, start with "Secure Your VPS Before Hackers Do" and the first Directus + Coolify post — the bundled, single-Compose-file setup — before following along with this one. Introduction In the first method, we coupled all of the services into one stack using a single Docker Compose file. The network between all services was created automatically, and we didn't have to start them up individually — which removes the risk of a race condition if service startup isn't handled properly. If you're running a single app, that's genuinely the recommended way to set up Directus on a Coolify-managed VPS. Going in, I assumed there were several good reasons to split the services apart instead — more control over backups, monitoring, restarts, that kind of thing. So before recommending decoupling, I actually tested each of those assumptions on a live Coolify instance. Most of them turned out to be wrong. Myth 1: Restarting Directus Restarts the Whole Stack I expected that restarting Directus inside the bundled Compose file would restart Redis and Postgres along with it. It doesn't. Coolify lets you restart each service in the stack independently — Directus, Database, and Cache each get their own Restart button, right there in the same view. No decoupling needed for this one. Myth 2: You Need a Separate Database Resource for S3 Backups Same story. Even with Postgres bundled inside the Directus Compose file, Coolify still gives it its own dedicated Backups option, S3 included. This isn't a separate-resource-only feature. Myth 3: Scheduled Tasks Require Separate Services Also not true. Coolify exposes a Scheduled Tasks tab per service, even inside a single bundled stack — complete with a Container name dropdown letting you target the cron job at just the database, or just Directus, without splitting anything apart. What Actually Holds Up Two things survived testing. First: metrics. This one's confirmed directly in Coolify'

If your files already live in Amazon S3, the pitch for storage that branches with your database is appealing but the word "migration" makes it sound like a project. It mostly is not. Neon's object storage speaks the S3 API, so the code you already wrote, the AWS SDK calls and presigned URLs, keeps working. What changes is how you point the client and where the bucket comes from, and that is a small, mechanical diff. The actual data move is a copy loop you can run once. The one thing to do up front is confirm the object operations your app actually relies on: the demo here exercises PutObject , GetObject , listing, and presigned URLs, and I flag the S3 features you should check for yourself further down. This post is the practical version: what stays identical, the exact config that changes, a script to copy the objects across, and an honest list of the S3 features that do not have an equivalent so you know what to check before you commit. The repo with the working client is at the end. TL;DR Neon object storage is S3-compatible. Your @aws-sdk/client-s3 code for the common operations, PutObject , GetObject , getSignedUrl , listing, works unchanged (these are what the demo verifies). Confirm anything beyond that, like multipart for large objects, against the current preview. The diff is the client config: point endpoint at the Neon storage endpoint, pin region: 'us-east-2' , set forcePathStyle: true . The bucket is declared in neon.ts instead of created in the console, and credentials are injected per branch. Move the data with a list-and-copy loop between two S3 clients (source AWS, destination Neon). What does not carry over: S3 bucket policies, event notifications and Lambda triggers, storage classes and Glacier transitions, and cross-region replication. Object CRUD and presigning do. The payoff is everything else in this series: once the files are on Neon, they branch with your database. Prerequisites An existing S3 bucket and credentials that can read it A Neon p

If your app stores files and you want real preview environments, you eventually hit the same wall: each preview needs its own storage, so you start provisioning a bucket per environment. That sounds cheap until you write it down. For every ephemeral environment you create a bucket, attach a policy, mint an IAM role or access keys, set CORS, add a lifecycle rule so it eventually cleans up, wire the credentials into the preview's config, and register a teardown step for when the PR closes. Then you find the orphaned buckets the teardown missed, months later, still billing. The reason this is painful is that the bucket is a separate resource from the database, so it needs its own lifecycle. Neon collapses that: the bucket is declared as part of the branch, so it is created and destroyed with the branch and needs no per-environment provisioning at all. This post compares the two approaches and shows the branch version working with no bucket-management code in sight. The repo is at the end. TL;DR Isolated storage per preview usually means provisioning a bucket per environment: policy, IAM, CORS, lifecycle, credential wiring, teardown. It is slow, it drifts, and it leaves orphaned buckets that keep costing money. On Neon the bucket is declared once in neon.ts . Creating a branch brings the bucket (with a copy-on-write copy of the files) and injects scoped credentials; deleting the branch removes it. There is no per-environment bucket to create, no IAM role to mint, and nothing to orphan. Copy-on-write means fifty preview buckets do not cost fifty times the storage, only what each one changes. Prerequisites A Neon project on the platform preview (object storage, us-east-2 ) The Neon CLI, and a CI system that opens/closes preview environments Familiarity with S3 buckets and IAM if you have done the manual version The per-environment bucket, written out Here is what "just give the preview its own bucket" actually expands to, per environment: Create a bucket with a unique nam

The naive way to accept file uploads is to POST them to your API, let the server read the bytes, and write them to object storage. It works until the files get large or the traffic gets real. Now every upload crosses your infrastructure twice, once from the client to your server and once from your server to storage, and your server holds the whole file in memory or on disk while it does. On a serverless function it is worse, because functions have request-size and duration limits that a big upload runs straight into. Presigned URLs are the standard fix, and they predate serverless by a decade. Your server does not move the bytes; it hands the client a short-lived, pre-authorized URL and the client uploads directly to object storage. The server only issues permission and records metadata. On a Neon Function this is the same AWS S3 SDK you already use, pointed at the branch's storage endpoint. This post builds it and tests the whole round trip. The repo is at the end. TL;DR Proxying uploads through a function sends the bytes across it, burning bandwidth and memory and hitting request-size limits. A presigned URL is a time-limited, pre-authorized link to one object key. The client PUTs the bytes straight to storage; the function never touches them. On Neon Functions you generate it with getSignedUrl from @aws-sdk/s3-request-presigner , the same code as any S3-compatible store. I tested the full flow: presign, the client PUT straight to storage returned 200 , a metadata record was saved, and downloading the object returned the exact bytes. One gotcha to pin: the injected AWS_REGION is the storage-cell host, not a region, so set region: 'us-east-2' on the client. Prerequisites A Neon project on the platform preview with a declared bucket (object storage, us-east-2 ) The AWS SDK: @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner Familiarity with S3-style object storage and HTTP PUT Why not just proxy the upload Sending the file through the function has three costs that

Step-by-step Xero API integration: OAuth 2.0, tenant routing, paging, rate limits, the 2026 scope and pricing changes, plus a no-code path to PostgreSQL. By Ilshaad Kheerdali · 4 August 2026 The Xero API is well documented and pleasant to work with once it clicks, but the first integration always takes longer than people expect. There is an extra discovery step that most accounting APIs don't have, tokens expire faster than you'd guess, and 2026 brought two changes that alter how you scope and budget an integration. This guide walks the whole flow: creating an app, running OAuth 2.0, resolving which organisation you're actually talking to, making your first call, paging through results, staying inside the rate limits, and pulling incremental updates. At the end it covers what changed in 2026 and the shortcut if the plumbing isn't the part you want to own. Everything below targets the Xero Accounting API over OAuth 2.0. Xero retired OAuth 1.0a some years ago, so any tutorial you find that mentions consumer keys and signed requests is out of date. What the Xero API Is The Xero Accounting API is a REST API that returns XML by default and JSON if you ask for it. You read and write accounting entities: Invoices , Contacts , Payments , BankTransactions , Accounts , CreditNotes , Items , PurchaseOrders , ManualJournals and a few dozen more, plus a set of report endpoints. The thing that surprises most developers coming from Stripe or QuickBooks is the tenant model . A single Xero login can have access to many organisations: an accountant might be connected to two hundred client orgs. So authorisation and targeting are two separate concerns. Your token proves the user said yes, and a separate header tells Xero which organisation the call is for. That means every integration has a step that a Stripe integration simply doesn't: after you get a token, you have to ask Xero which tenants that token can reach. Step 1: Create a Xero App Sign in at the Xero Developer portal and cre

A generated image looks finished until review starts. Someone approves the first version. Someone else crops it. A branded copy goes out. Another edit changes the prompt. A week later, the useful question is simple: which prompt, model, seed, size, parent image, and publishing settings produced the version on screen? In a content studio, I put those answers in the PostgreSQL row that stores the image. Logs explain what happened during a run, then rotate away. Object storage keeps the bytes and forgets why they exist. The row is the only one of the three that survives edits, review, and publishing. 1. The row is the receipt The table in apps/api/src/database/init-ai-images-table.js treats generated and edited images as one record type. An original image gets its own row. An edit gets another row, with original_image_id pointing back to the parent. CREATE TABLE IF NOT EXISTS ai_generated_images ( id SERIAL PRIMARY KEY , image_url TEXT NOT NULL , -- what produced it prompt TEXT NOT NULL , model VARCHAR ( 100 ) DEFAULT 'fal-ai/imagen4' , model_version VARCHAR ( 100 ), seed BIGINT , width INTEGER , height INTEGER , -- how it derives from another row is_edited BOOLEAN DEFAULT FALSE , original_image_id INTEGER REFERENCES ai_generated_images ( id ), edit_prompt TEXT , edit_strength DECIMAL ( 3 , 2 ), -- what actually shipped branded_url TEXT , branding_options JSONB , metadata JSONB , tags TEXT [], created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); That self-reference is the design choice. It makes the image table append-only-ish: new variants are inserted as new rows instead of overwriting the earlier state. The cost is more rows and more discipline at write time. The benefit is editable history that product screens and debugging queries can follow. flowchart TD original["Original row: prompt, model, seed, dimensions"] editA["Edited child: edit_prompt, edit_strength, edit_steps"] editB["Edited child: edit_prompt, edit_guidance_scale"] brandedA["Branded output: branded_url,

Originally published at nlqdb.com/blog The standard agent-memory build is an afternoon of work: embed every fact worth keeping, upsert it into a vector store, and before each reply pull the top-k most similar memories back into context. And for what it's built for, it works. Ask "what did this user say about the Berlin migration" and the right snippets come back, ranked by cosine distance. Recall is solved enough that it feels like memory is solved. Then the agent has been running for a month, and you ask its memory a different kind of question: "how many users asked about pricing this month?" "Average deal size per stage?" "Top 10 topics I logged, ranked by count?" The store dutifully returns the twenty memories most similar to the question text , the LLM eyeballs them, and you get a confident, specific, wrong number. Recall is similarity. Reporting is aggregation. Nothing malfunctioned — the two questions want different machines. A vector store's primitive is nearest-neighbour search: embed the query, rank stored vectors by distance, return the top-k, optionally narrowed by a metadata filter. That is the whole contract. There is no COUNT , no GROUP BY , no JOIN , no HAVING — a similarity engine ships no query planner, and even the metadata filter only narrows candidates around the approximate search, so what comes back is still a ranking of similar items, never a computed result set. "How many" has to touch every matching row . If the agent logged 4,000 memories and top-k is 20, the context the LLM sees is structurally incapable of producing the count — and an LLM doing arithmetic over a retrieved sample is a hallucination generator, not a query engine. The failure is quiet, too: the answer arrives fluent and plausible, and nothing flags that it was computed from half a percent of the data. -- "top topics this month, ranked by count" is not a similarity query. -- It's this — and it must scan every matching row, not the top-k: SELECT topic , count ( * ) AS mentions

Disclosure: I build Schemity , a desktop ERD tool - this post is from our blog and uses it for the examples. TL;DR: Database views carry real responsibilities - reporting layers, security boundaries, API surfaces - but ERD tools either leave them out entirely (DBML has no view support despite requests since 2022) or draw them as if they were ordinary tables. Schemity displays views and materialized views as read-only entities with italic names and a bold view or mview token in the entity footer, so derived relations are distinguishable from base tables at a glance, and they can be imported into context views like any other entity. A database view belongs in your ERD, but not disguised as a table: it is a derived, read-only relation, and the diagram should say so at a glance. Schemity draws views and materialized views as read-only entities with italic names - present on the canvas, visually distinct from the base tables they are built on. That sentence would be unremarkable if the rest of the tooling world agreed with it. Mostly, it does not. In most ERD tools your views are simply absent, and in the rest they are dressed up as something they are not. The reporting layer your diagram pretends does not exist Views are not decoration. They are where schemas put their public face: the reporting layer that joins five tables into one readable relation, the security boundary that exposes a subset of columns to an application role, the compatibility shim that survives a refactor. On Supabase , views are how you shape what PostgREST exposes as an API. A materialized view may be the single most performance-critical object in an analytics schema. Whoever reads your diagram to understand the system needs to see them. Yet the diagram usually cannot show them. DBML - the schema language behind dbdiagram.io - has no syntax for views at all: a user proposed designing views with join definitions in December 2022, others were still upvoting the request in July 2024, and there has be

TimescaleDB 2.27, released May 12 2026, extends bloom-filter batch pruning from reads to writes. UPDATE, DELETE, and UPSERT against compressed columnstore data can now skip decompressing batches that provably cannot contain the target rows. The reported gains are real: up to 160x for selective UPDATE/DELETE, and over 2x for UPSERT. The feature is automatic. Whether it is actually firing on your workload is not something you can assume, and the only way to confirm it is to read new EXPLAIN counters that the release notes mention but do not explain. Worse, the counter names are inconsistent between the write paths, so even a careful reader ends up guessing. This post is about reading those counters correctly, and about the two things in this release that will silently break a query if you upgrade without noticing them. What is actually being skipped A quick model of the mechanism, because the counters only make sense against it. Hypercore stores compressed data in batches, roughly a thousand rows each. For columns that are not the segmentby key, TimescaleDB maintains a sparse bloom filter per batch: a small probabilistic summary that answers one question, "could this batch contain column = X ?", without touching the compressed payload. A bloom filter has a useful asymmetry. A negative is certain: if the filter says no, the value is definitely absent, and the batch can be skipped whole. A positive is not: the filter says "maybe", you decompress, and sometimes the value is not there after all. That last case is a false positive, and it is the number that tells you whether the whole scheme is paying off. Before 2.27, a DELETE ... WHERE sensor_id = 'x' against compressed data decompressed every candidate batch to check. Now the bloom filter is consulted first, and batches that cannot match are never decompressed. The work you save is the decompression of the batches that get pruned. The work you waste, when the filter is poorly matched to your data, is the bloom check on

A Patroni cluster needs an odd number of nodes to maintain quorum — with 3 nodes, losing 1 still leaves a majority, so the cluster keeps running. This guide builds a 3-node PostgreSQL cluster on Ubuntu 24.04 with Patroni handling replication and automatic failover, etcd as the coordination store, and HAProxy load-balancing client connections — all secured with TLS. Prerequisites: three Ubuntu 24.04 servers (2 vCPU / 4GB RAM minimum) with PostgreSQL installed, non-root sudo access, and a domain with three A records: node1.example.com , node2.example.com , node3.example.com . Replace these placeholders with your actual subdomains throughout. Install Dependencies Run on all three nodes unless noted otherwise. 1. Install packages: $ sudo apt update $ sudo apt install haproxy certbot pipx -y $ sudo pip3 install --break-system-packages 'patroni[etcd3]' psycopg2-binary psycopg 2. Install etcd: $ wget https://github.com/etcd-io/etcd/releases/download/v3.6.4/etcd-v3.6.4-linux-amd64.tar.gz $ tar -xvf etcd-v3.6.4-linux-amd64.tar.gz $ sudo mv etcd-v3.6.4-linux-amd64/etcd etcd-v3.6.4-linux-amd64/etcdctl /usr/local/bin/ 3. Open firewall ports — 80 (Certbot), 2379/2380 (etcd), 5432/5433 (PostgreSQL + Patroni-managed PostgreSQL), 8008/8009 (Patroni REST API): $ sudo ufw allow 80,2379,2380,5432,5433,8008,8009/tcp $ sudo ufw reload $ sudo ufw status Configure SSL Certificates 1. Request a certificate per node (run on each node for its own subdomain): $ sudo certbot certonly --standalone -d node1.example.com -m admin@example.com --agree-tos --no-eff 2. Create a cert-prep script on each node (set HOSTNAME to that node's subdomain): $ sudo nano /usr/local/bin/prepare-ssl-certs.sh #!/bin/bash HOSTNAME = "node1.example.com" # Update for each node CERT_DIR = "/etc/letsencrypt/live/ $HOSTNAME " ARCHIVE_DIR = "/etc/letsencrypt/archive/ $HOSTNAME " getent group ssl-users > /dev/null || sudo groupadd ssl-users for user in etcd patroni haproxy postgres ; do if ! id " $user " > /dev/null 2>&1 &&

Liquid syntax error: Unknown tag 'endraw'

Why this combination keeps winning for IoT telemetry backends — not in a benchmark, but against flaky gateways, replayed data, and firmware that never quite agrees with itself. If you’ve ever built the backend for a fleet of IoT devices — sensors, gateways, industrial equipment reporting temperature, humidity, GPS, battery, signal strength — you’ve faced the same fork in the road early on: what do you build the ingestion layer with, and what do you store the data in? After building a telemetry backend from scratch for a real fleet of LoRa/BLE sensors and gateways — handling dual ingestion paths, binary and JSON payload formats, automatic recovery of lost data, and a growing set of operational dashboards — I keep coming back to the same combination: Python (FastAPI + asyncio) for the API, MQTT for device transport, and PostgreSQL for storage. Here’s why that combination holds up so well for this specific problem, not just “in general.” Full Article: https://medium.com/@jackpelorus/python-postgresql-and-mqtt-the-boring-stack-that-actually-survives-a-real-device-fleet-9297146cbe8d?sharedUserId=jackpelorus

SQL is quite flexible, making it easy to write a single query that works for two situations: one without a parameter and a WHERE clause, and another with a parameter for filtering, all in the same SQL query. For example, I came across a benchmark comparing MongoDB and PostgreSQL that shows how to handle pagination effectively—by avoiding OFFSET and instead using the last value to fetch the next set of results. The first page includes a WHERE clause along with ORDER BY and LIMIT, while the following pages add an extra WHERE condition. In the MongoDB version of this benchmark, the filter is handled within the application, which leads to two separate queries for these scenarios. export async function getOrders ( cursor ) { const match = cursor ? { timestamp : { $gt : new Date ( cursor ) } } : {}; const rows = await orders . aggregate ([ { $match : match }, { $sort : { timestamp : 1 } }, { $limit : PAGE_SIZE }, ]) We can do the same in PostgreSQL using a single prepared statement. SQL is such a powerful language that it often feels tempting to write it this way: SELECT * FROM orders WHERE $ 1 :: timestamptz IS NULL OR "timestamp" > $ 1 ORDER BY "timestamp" ASC LIMIT $ { PAGE_SIZE } If $1 is NULL, it skips the second condition in the OR clause and retrieves all rows without filters, resulting in a broad fetch. When $1 has a value, it filters the results using that specific value, enabling a more targeted search. However, using a generic query can sometimes lead to a less-than-ideal execution plan that's not perfectly tailored for each specific situation. I gave it a try: drop table if exists orders ; create table orders ( order_id text primary key , "timestamp" timestamptz not null ); create index idx_orders_timestamp on orders ( "timestamp" ); insert into orders select 'ORD-' || g , '2025-01-01' :: timestamptz + g * interval '1 minute' from generate_series ( 1 , 5000000 ) as g ; analyze orders ; prepare getorders ( timestamptz , int ) as select * from orders where $ 1 :

I've always been amazed by how Oracle Database handles updates to a unique column—performing set-based operations that don't violate the unique constraint, yet when executed row by row, it temporarily permits duplicates. SQL > create table franck ( val int unique ); Table created . SQL > insert into franck values ( - 1 ) , ( 1 ) ; 2 rows created . SQL > select val from franck ; VAL ---------- - 1 1 SQL > update franck set val =- val ; 2 rows updated . SQL > select val from franck ; VAL ---------- 1 - 1 From a SQL perspective, this is expected behavior, but not all databases support it without raising an error: Db2 , SQL Server , and Oracle handle it without error. PostgreSQL raises ERROR: duplicate key value violates unique constraint "franck_val_key", DETAIL: Key (val)=(1) already exists. This works with a deferred constraint. MySQL or MariaDB raise Duplicate entry '1' for key 'franck.val' SQLite raises { "code": "SQLITE_CONSTRAINT_UNIQUE" } MongoDB raises E11000 duplicate key error collection: test.franck index: val_1 dup key: { val: 1 } db . franck . createIndex ({ val : 1 }, { unique : true }); db . franck . insertMany ([ { val : - 1 }, { val : 1 } ]); db . franck . updateMany ({},[ { $set : { val : { $multiply :[ " $val " , - 1 ]} } } ]); MongoServerError : Plan executor error during update :: caused by :: E11000 duplicate key error collection : test . franck index : val_1 dup key : { val : 1 } This is surprising because Oracle unique indexes store the indexed columns as the B-tree key and the ROWID as the associated data. Non-unique indexes add the ROWID to the physical key and are required for a deferrable unique constraint to allow temporary duplication before the end of the transaction. So how do non-deferrable unique indexes allow duplication during a single update statement? In this simple example, I would expect: The initial index entries are: (-1): row #1 and (1): row #2 Updating the first row deletes the first entry (-1): row #1 and adds one with (1):