标签：#data

BadHost is a high-severity authentication bypass vulnerability in the widely used Python web framework Starlette, with 325 million weekly downloads. The flaw allows attackers to use malformed HTTP Host headers to bypass path-based access controls and access sensitive AI agent infrastructure, among other systems. By Sergio De Simone

Crescent Island is an air-cooled chip that uses LPDDR5 memory.

Introduction In contemporary research, the volume of biological data deposited in public repositories is growing exponentially. The Gene Expression Omnibus (GEO), NCBI Gene, PubMed, and UniProt accumulate thousands of new records daily, including sequences, expression profiles, scientific articles, and functional annotations. On the one hand, this scenario represents a unique opportunity for biomedical research. On the other hand, the diversity of data formats, access protocols, and metadata models creates a significant barrier: each source requires a specific collector, distinct rate-limiting strategies, and its own validation logic. Above all, the lack of standardization in data storage compromises the reproducibility of scientific studies. The need for integrated tools capable of unifying data extraction, curation, and persistence has been widely discussed. In practice, ad hoc solutions such as isolated scripts for individual repositories generate redundant work and make maintenance difficult. First and foremost, it is necessary to establish an architecture that treats data collection as a service rather than a collection of scattered artifacts. This work presents Project 1 of the Integrated Bioinformatics Platform: a containerized Biomedical Data Collector coupled with a Data Lake. Its objective is to provide a REST API capable of triggering asynchronous data collections from the four aforementioned sources, storing immutable raw data in MinIO, and persisting metadata in PostgreSQL, all while ensuring traceability and resilience. Development The system architecture is divided into three main layers. The first is the API and orchestration layer , implemented using FastAPI. Its five endpoints — POST /collections , GET /collections , GET /collections/{id} , GET /collections/{id}/download/{dataset_id} , and GET /health — expose a clean interface for initiating and monitoring collection processes. The second layer is the collector engine , composed of abstract classe

Strava will charge a flat monthly fee from developers to access its API

Modern search engines do more than match keywords. When you search for "cozy mystery set in Paris" and get results for "atmospheric detective novel in France" that's vector search at work: documents and queries are converted into lists of numbers, called embeddings, and the search engine finds the documents whose numbers are closest to the query's. Manticore Search supports this natively. Under the hood, it uses a data structure called HNSW: a graph that connects nearby vectors, so it can find nearest neighbors quickly without scanning every document. That makes vector search fast enough to run on millions of documents in milliseconds. But HNSW has an inefficiency. Early in the traversal, almost every distance computation finds a better candidate than the ones already in the result set. As the search goes on, those improvements become rarer, but the algorithm keeps traversing the graph until it exhausts its exploration budget. By that point, the result set has often already converged, and the remaining work does little or nothing to improve it. Early termination fixes this by detecting that point and stopping early. The effect becomes more noticeable as k grows, where k is the number of nearest neighbors the query asks Manticore to return. Returning more neighbors requires more graph exploration, and much of that extra work happens after the result set has already stabilized. That also makes early termination more valuable, because it has more unnecessary work to cut. This gets more pronounced with vector quantization . Quantization compresses stored vectors to save memory, which slightly lowers search precision. To recover it, Manticore uses oversampling : it fetches 3x more candidates than requested, then rescores them using the original full-precision vectors. With the default 3x oversampling, HNSW explores many more candidates per query. Large k values often come from this kind of candidate expansion: an application may ask the vector index for hundreds or thous

0A000 feature not supported 는? PostgreSQL 에러 코드 0A000 은 현재 사용하려는 기능이 PostgreSQL에서 지원되지 않거나, 특정 컨텍스트에서는 사용할 수 없음을 의미합니다. 주로 트랜잭션 내부에서 허용되지 않는 명령을 실행하거나, 해당 버전의 PostgreSQL에서 아직 구현되지 않은 SQL 표준 문법을 사용할 때, 또는 복제(Replication) 환경의 제약으로 인해 발생합니다. 실무에서는 특히 CREATE DATABASE , VACUUM , CLUSTER 같은 명령을 트랜잭션 블록 안에서 실행하거나, Logical Replication 슬롯과 관련된 작업을 수행할 때 자주 마주치는 에러입니다. 주요 발생 원인 1. 트랜잭션 블록 내에서 허용되지 않는 DDL 명령 실행 PostgreSQL은 일부 DDL 명령을 트랜잭션 블록( BEGIN ... COMMIT ) 내에서 실행하는 것을 허용하지 않습니다. CREATE DATABASE , DROP DATABASE , CREATE TABLESPACE , DROP TABLESPACE , VACUUM , CLUSTER 등의 명령은 트랜잭션 컨텍스트 밖에서 단독으로 실행되어야 하며, 이를 무시하고 트랜잭션 내부에서 호출하면 0A000 에러가 발생합니다. 2. 특정 PostgreSQL 버전에서 지원하지 않는 문법 또는 기능 사용 SQL 표준에는 정의되어 있지만 PostgreSQL의 해당 버전에서 아직 구현되지 않은 기능을 사용할 때 이 에러가 발생합니다. 예를 들어, 구버전 PostgreSQL에서 LATERAL JOIN , MERGE 문, GENERATED ALWAYS AS (expression) STORED 컬럼 정의 등을 사용하거나, 특정 윈도우 함수 옵션 조합을 사용하는 경우 해당 에러를 만날 수 있습니다. 3. 논리 복제(Logical Replication) 또는 스트리밍 복제 환경에서의 제약 위반 Standby 서버나 Logical Replication 구독자(Subscriber) 측에서 쓰기 작업이나 특정 관리 명령을 실행하려 할 때 0A000 에러가 발생합니다. Hot Standby 상태의 서버에서 DDL을 실행하거나, Logical Replication 슬롯이 활성화된 상태에서 지원되지 않는 방식으로 복제 슬롯을 조작하려는 경우 이 에러를 마주치게 됩니다. 해결 방법 원인 1: 트랜잭션 블록 내 허용되지 않는 DDL 실행 트랜잭션 블록을 제거하고 해당 명령을 단독으로 실행하는 것이 핵심입니다. 아래는 잘못된 예제와 올바른 예제를 비교한 것입니다. ❌ 잘못된 예 (0A000 에러 발생) BEGIN ; CREATE DATABASE myapp_db WITH OWNER = myapp_user ENCODING = 'UTF8' LC_COLLATE = 'ko_KR.UTF-8' LC_CTYPE = 'ko_KR.UTF-8' ; COMMIT ; -- ERROR: 0A000: CREATE DATABASE cannot run inside a transaction block ✅ 올바른 예 (트랜잭션 블록 밖에서 실행) -- 트랜잭션 블록 없이 단독 실행 CREATE DATABASE myapp_db WITH OWNER = myapp_user ENCODING = 'UTF8' LC_COLLATE = 'ko_KR.UTF-8' LC_CTYPE = 'ko_KR.UTF-8' ; VACUUM 명령도 동일한 원칙이 적용됩니다. -- ❌ 잘못된 예 BEGIN ; VACUUM ANALYZE public . orders ; COMMIT ; -- ✅ 올바른 예: 트랜잭션 밖에서 단독 실행 VACUUM ANALYZE public . orders ; -- ✅ 특정 테이블만 선택적으로 VACUUM VACUUM ( VERBOSE , ANALYZE ) public . orders ; 애플리케이션 코드(예: Python psycopg2)에서 자동 커밋을 끈 상태로 VACUUM을 호출하는 경우도 흔한 실수입니다. import psycopg2 conn = psycopg2 . conn

Environmental activist Erin Brockovich has a new mission.

How a Product Sync Automation Project Transformed Customer Onboarding When people think about impactful engineering work, they often imagine distributed systems, high-scale infrastructure, or complex algorithms. One of the most impactful projects I worked on wasn't any of those. It was solving a seemingly simple problem: Keeping product data in sync across multiple retail systems. Years later, our CEO still remembers how much smoother customer onboarding became after this project. The Context: What is Commerce Connect? At Casa Retail AI, we have an internal platform called Commerce Connect (CC) . Commerce Connect acts as the central Product Information Management (PIM) system and serves as the source of truth for product information. Under the hood, it is built on top of a customized version of the open-source e-commerce platform Spree Commerce , extended with multi-vendor and multi-tenant capabilities. Its primary responsibility is simple: Collect product information from multiple retail ecosystems and distribute it to every Casa product that needs it. Once product data enters Commerce Connect, it is synchronized to multiple downstream systems. Why Product Data Matters Many applications inside Casa depend on product information. Product Consumers Once product data enters Commerce Connect, it is distributed to multiple systems across the Casa ecosystem. Customer-Facing Applications Several products rely on product information to provide context and improve customer experience: Lead management applications use product information during customer interactions. Ticket management systems link customer issues to specific products. Digital receipts display product names, images, and related details. Analytics & Reporting Product data powers business dashboards and reports, helping retailers answer questions such as: Which categories perform best? Which products attract the most attention? Which products generate the most complaints? It is also used for filtering and segme

Now that almost everyone has thought about or is actively integrating AI workflows into their projects, some might ask is this all worth the cost? Many think the current economics of the AI space don't scale and that there will be upward price movement. Others still might not be comfortable with sending their data to remote services for processing. Then there is the crowd that wants to deploy models in small spaces with limited compute. Are there ways we can deploy small models locally and run at a lower cost? Yes with Knowledge Distillation . Knowledge distillation can get a bad rap due to it's questionable use in training some Large Language Models (LLMs). But it's a perfectly valid way to transfer performance from a larger model to a smaller one. Especially when both models are yours and/or open. This article will explore progressive distillation which is a technique to incrementally transfer knowledge from a series of larger teacher models into a smaller student. Install dependencies Install txtai and all dependencies. pip install txtai [ pipeline - train ] datasets Setup the Training Pipeline The first step we need to do is setup up the training pipeline. We'll use the Hugging Face Training framework to build a series of models. The following code establishes a train method, test method and loads the classification training data. from datasets import load_dataset from transformers import AutoModelForSequenceClassification , AutoTokenizer from txtai.pipeline import HFTrainer , Labels def train ( teacher , student , distillation , ** kwargs ): trainer = HFTrainer () model = AutoModelForSequenceClassification . from_pretrained ( student , trust_remote_code = True ) tokenizer = AutoTokenizer . from_pretrained ( student , trust_remote_code = True ) return trainer ( ( model , tokenizer ), ds [ " train " ], columns = ( " sentence " , " label " ), maxlength = maxlength , teacher = teacher , distillation = distillation , ** kwargs ) def test ( model ): labels = Labels (

The Theorem That Changed How We Think About Databases In 2000, Eric Brewer stood at a conference and proposed a conjecture that would reshape distributed systems forever: "You can only guarantee two of these three properties at the same time: Consistency, Availability, and Partition Tolerance." Two years later, Seth Gilbert and Nancy Lynch proved it mathematically. It became known as the CAP Theorem — and every distributed system architect since has had to wrestle with it. It sounds abstract. But once you understand it, you'll never look at a database choice the same way again. You'll understand why Amazon DynamoDB and Google Spanner make opposite architectural choices. You'll know why your bank uses PostgreSQL while Twitter uses Cassandra. Let's break it down from first principles. The Three Properties C — Consistency Every read receives the most recent write, or an error. There's only one version of the truth — all nodes agree. Not the same consistency as ACID . CAP consistency (linearizability) means every read reflects the latest write across all nodes. ACID consistency means transactions don't violate database constraints. Different concepts, same confusing word. A — Availability Every request receives a non-error response — though it might not be the most recent data. The system is always up and answering. Note: "Available" in CAP doesn't mean "fast." It means "responds without error." A system that always returns a (possibly stale) answer is Available. P — Partition Tolerance The system continues operating even when network messages between nodes are lost or delayed. A partition is when part of your distributed system can't communicate with another part. The Unavoidable Truth: P Is Not Optional Here's the insight that makes CAP actually useful: In any real distributed system, partitions will happen. Networks fail. Cables get cut. Data centers lose connectivity. AWS regions go down. Since you must tolerate partitions (or have a single-server system, which does

DuckDB has recently announced Quack, a new remote protocol over HTTP that lets multiple DuckDB instances connect to and work with the same database over a network. The protocol introduces client-server capabilities to a database that was previously mostly local and embedded. By Renato Losio

AI applications are moving quickly from simple chatbots to systems that can search, reason, recommend, summarize, and act on live business data. For developers, that usually means wiring together databases, embedding models, vector search, rerankers, orchestration logic, and application code. For no-code AI builders, it often means waiting for those integrations to exist before an idea can become a working prototype. The MongoDB extensions for Dify help close that gap. With the new MongoDB Atlas and Voyage AI extensions, Dify builders can visually compose AI workflows and agents that connect directly to MongoDB data, perform semantic retrieval with Atlas Vector Search, improve result quality with Voyage AI embeddings and reranking, and optionally interact with operational documents through controlled database tools. The result is a practical path from idea to working AI application: less custom orchestration code, more reusable building blocks, and a smoother experience for both developers and no-code builders. Why Dify and MongoDB Belong Together Dify provides a visual environment for building AI apps, workflows, and agents. It makes it easy to connect user input, model calls, tools, prompts, and outputs into a working application. MongoDB Atlas provides the data foundation: flexible documents, operational queries, aggregation, full-text search, and vector search in one platform. Together, they create a powerful pattern: Dify orchestrates the AI experience — workflows, agents, prompts, tools, and user interactions. MongoDB Atlas stores and retrieves the data — documents, application records, knowledge sources, and vector embeddings. Voyage AI improves retrieval quality — embeddings for semantic search and reranking for precision. For a no-code builder, this means you can assemble a retrieval-augmented generation workflow visually. For a developer, it means the integration points are packaged as reusable Dify tools rather than one-off glue code. Meet the Extensions

INTRODUCTION Configuring a Power BI semantic model involves refining data structures, creating relationships, and setting up calculations. Semantic model is the last stop in the data pipeline before reports and dashboards are built. It is the end product of the raw data that has been extracted, transformed, loaded, modeled, built relationship, and written calculation. The Semantic model consist of Data connections to one or more data sources, Transformations that clean and prepare the data for reporting, Defined calculations and metrics based on business rules to ensure consistent reports and Defined relationships between tables. Key words to note in Semantic Modelling are; 1. Fact table and Dimension table: The Fact table records the quantitative and numerical data. It is where every single details are recorded. The Dimension table act as the descriptive companion to the fact table, containing the attributes or characteristics that provide context to the data. 2. Primary and Foreign Key: Primary Keys are unique identifier assigned to a specific record with a database table ensuring that no two rows are identical or repeated. foreign Keys are columns or group of columns in one table that provides a link between data in two tables by referencing the primary key of another. 3. Star Schema Star Schema is a data modeling technique where a central fact table is surrounded by several dimension tables that provide descriptive content. 4. Cardinality Cardinality defines the kind of relationship between two tables. They are; One to Many (1.*) Many to one (*.1) One to One (1.1) Many to Many ( . ) The cardinality of a relationship is described by the "one" (1) or "many" (*) icons located at the ends of the relationship line. 5. Cross Filter Direction The direction determine how filters propagate. Possible cross filter options are dependent on the relationship cardinality type. One to Many - Single or Both sides One to One - Both sides Many to Many - Single to either table or b

The honest decision framework, not another flame war. The SQL vs NoSQL debate has been running for 15 years and it still generates more heat than light. Here's the framework that actually helps you decide. The Real Question It's not "SQL or NoSQL." It's: what does your access pattern look like? If your application is mostly reading and writing related data through well-defined queries — orders with line items, users with addresses, products with categories — relational databases are purpose-built for this. JOINs are not expensive when they're indexed. Transactions are not slow when they're scoped correctly. PostgreSQL handles 50 million rows comfortably on a single node. If your application is reading and writing self-contained documents with predictable access by a primary key, and you rarely need cross-document queries — user profiles, product catalogs, content management — a document database simplifies your code. No ORM mapping hell. No migration files for adding a field. If your application writes massive volumes and reads by partition key with eventual consistency — time-series data, IoT telemetry, activity feeds at scale — wide-column stores like Cassandra were built for this specific workload. The 2026 Reality PostgreSQL has eaten NoSQL's lunch in many areas. JSONB support means you can store and query unstructured data inside PostgreSQL with GIN indexes. You get the document model flexibility without giving up transactions, JOINs, and a 30-year ecosystem. For 80% of startups and mid-size companies, PostgreSQL is the only database you need. MongoDB has gotten more relational. Multi-document ACID transactions (since 4.0), schema validation, aggregation pipelines that look suspiciously like SQL. It's converging toward what PostgreSQL already does, but with a different starting point. DynamoDB dominates serverless. If you're in AWS and your access pattern is simple key-value with known query patterns, DynamoDB's pricing model (pay-per-request) and operational s

Inspired by a similar project called GenosDB and Cloudflare’s initiative to rebuild Next.js, I decided to rebuild GunDB with a modern coding style, incorporating improvements and addressing shortcomings in the original technology. I used the OpenCode tool with the Big Pickle model to rewrite the project in a new graph database called Garfo (the Portuguese word for “fork”), and I was impressed with the results and its practical applications. In this article, I’ll explain the technology and its improvements over GunDB. Introduction Garfo is a modern, browser-first fork of GUN.js — the decentralized, offline-first graph database. A fork of the original project that keeps the familiar GUN graph API while bringing meaningful improvements to the modern JavaScript ecosystem. Why a Fork? GUN.js is a revolutionary technology — a graph database that syncs in real time, works peer-to-peer, resolves conflicts automatically, and runs in the browser. However, the JavaScript ecosystem has evolved. TypeScript has become the standard, ES modules are the norm, and new transport layers like Nostr have emerged as promising decentralized protocols. Garfo was born to fill these gaps: a GUN rewritten with modern typing, designed with the browser as a first-class citizen, and with native support for the Nostr protocol. Key Features Familiar Graph API If you've used GUN before, you'll feel right at home. Garfo exposes the same chainable API: import Garfo from ' garfo ' ; const db = new Garfo ({ localStorage : true }); db . get ( ' users ' ). get ( ' alice ' ). put ({ name : ' Alice ' , status : ' online ' }); db . get ( ' users ' ). get ( ' alice ' ). on ( profile => { console . log ( ' Update: ' , profile ); }); All the classic methods are there: get() , put() , set() , on() , once() , map() . Optional Nostr Transport This is one of the most exciting additions. Garfo can use Nostr relays as a transport layer, allowing peers to exchange graph messages through public or private relays: const

Happy to announce HeliosProxy !! Far beyond a pooling tool, HeliosProxy ** is a next-gen programmable Postgres data-plane. **Works with PostgreSQL-compatible databases , not only HeliosDB. It starts as a PgBouncer-compatible wedge, then adds the operational surface teams usually build from multiple tools: connection pooling failover and transaction replay shadow execution anomaly detection edge cache controls admin REST API embedded admin UI signed WASM plugins OCI-style plugin artifacts Kubernetes operator Terraform and Pulumi providers 22 installable Claude/Codex operator skills Install operator skills: heliosdb-proxy install skills PostgreSQL #DevOps #SRE #Database #AIcoding

Event sourcing gets sold on immutability. You don't update, you don't delete, you only append, so the history is permanent. It mostly isn't. The events are immutable because your code agrees not to touch them, not because anything actually stops it. Underneath they're still rows in Postgres, and rows have a DBA with write access. A migration that "cleans up" old data. A 2 a.m. query run against the wrong connection. A backup restored with slightly different bytes in it. Change one of those rows and a replay won't blink. The aggregate rebuilds, the projections rebuild, everything looks fine. Usually the first person to notice is a customer whose balance is off, and by then the trail is cold. Chain each event into the next The trick is small. Give every row two extra columns: a hash of its contents, and the hash of the row before it. #1 AccountOpened prev=00000… hash=70be4f… │ ▼ #2 AmountDeposited prev=70be4f… hash=796018… │ ▼ #3 AmountWithdrawn prev=796018… hash=6a0260… The hash is SHA-256(previousHash || json(payload)) . Nothing exotic. The point is that each hash depends on the one before it. Edit a payload and its hash stops matching. Rewrite that hash to cover for the edit, and now the next row's pointer is wrong. You can't fix one without breaking the next. About forty lines of it Appending an event hashes it together with the previous one: public HashChainedEntry Append ( object payload ) { var previousHash = _entries . Count == 0 ? GenesisHash : _entries [^ 1 ]. Hash ; var hash = ComputeHash ( previousHash , payload ); var entry = new HashChainedEntry ( _entries . Count + 1 , payload , previousHash , hash ); _entries . Add ( entry ); return entry ; } internal static byte [] ComputeHash ( byte [] previousHash , object payload ) { var payloadJson = JsonSerializer . SerializeToUtf8Bytes ( payload , payload . GetType ()); var combined = new byte [ previousHash . Length + payloadJson . Length ]; Buffer . BlockCopy ( previousHash , 0 , combined , 0 , previousHash .

Si vous utilisez PostgreSQL, vous avez probablement déjà dû choisir entre une clé primaire BIGSERIAL et un UUID. Depuis des années, la version 4 (aléatoire) est le choix par défaut quand on veut un identifiant unique et distribué. Mais en 2026, une alternative plus récente s’impose : UUID v7, qui intègre un timestamp et promet de meilleures performances pour les index. Dans cet article, je vous explique concrètement ce qui change, avec des benchmarks PostgreSQL et des exemples de code, pour que vous puissiez décider en connaissance de cause. UUID v4 : le standard aléatoire et son problème d’index Un UUID v4 est constitué de 122 bits aléatoires. Cette absence totale de tri est sa force pour l’unicité, mais elle devient un handicap dans un index B‑tree, qui est la structure utilisée par PostgreSQL pour les clés primaires. Lorsque vous insérez un nouvel UUID v4, il a autant de chances de se retrouver au début de l’index qu’à la fin. Résultat : l’index se fragmente, les pages se remplissent mal, et les performances d’écriture se dégradent à mesure que la table grossit. J’ai reproduit un test simple sur PostgreSQL 16 avec 10 millions de lignes, en utilisant une table dont la seule différence est la colonne id : -- Table UUID v4 CREATE TABLE events_v4 ( id UUID DEFAULT gen_random_uuid () PRIMARY KEY , payload JSONB , created_at TIMESTAMPTZ DEFAULT now () ); -- Table UUID v7 (généré côté application, voir plus bas) CREATE TABLE events_v7 ( id UUID PRIMARY KEY , payload JSONB , created_at TIMESTAMPTZ DEFAULT now () ); Après insertion, voici les mesures : Type de clé Taille de l’index Fragmentation Latence moyenne d’insertion BIGINT ~214 Mo 0 % ~0,8 ms/ligne UUID v4 ~428 Mo (2×) 99 % ~4,8 ms/ligne UUID v7 ~428 Mo (2×) ~2 % ~1,1 ms/ligne Ce qui frappe, c’est la fragmentation quasi nulle de l’UUID v7. L’index reste compact et les insertions sont presque aussi rapides qu’avec un BIGSERIAL. L’UUID v4, lui, est plus de quatre fois plus lent à l’insertion sur ce volume. UUID v7 :

The engineering team at Meta recently outlined how the company migrated a data ingestion platform that transfers several petabytes of MySQL social graph data daily to improve reliability and operational efficiency. The team used techniques like reverse shadowing and continuous checksum monitoring to ensure zero downtime during the transition. By Renato Losio

Thinking in terms of existence instead of lists SQL Pattern Series #1 of 21 A collection of practical SQL patterns that help developers recognize common solutions to recurring database problems. What You'll Learn In this article you'll learn: When EXISTS and IN solve the same problem The difference between set membership and existence Why the underlying mental model matters When I typically reach for EXISTS Most SQL developers write a query like this at some point: SELECT c . CustomerID , c . CustomerName FROM Customers c WHERE c . CustomerID IN ( SELECT o . CustomerID FROM Orders o ); And it works. But sometimes it isn't the best way to think about the problem. The Question Behind the Query Many SQL problems can be framed in two different ways. Set Membership Is this value in a set? WHERE CustomerID IN (...) Existence Does at least one matching row exist? WHERE EXISTS (...) Both approaches often return the same result. But they represent different mental models. The Presence Pattern The Presence Pattern is useful when you do not actually care about the values being returned from a related table. You only care whether a matching row exists. For example: Customers who have placed an order Users who have logged in Employees assigned to a project Products that have sales In these cases, the question is often: Does a related row exist? rather than: What values are contained in this list? Example Using EXISTS SELECT c . CustomerID , c . CustomerName FROM Customers c WHERE EXISTS ( SELECT 1 FROM Orders o WHERE o . CustomerID = c . CustomerID ); The subquery is correlated to the outer query. Conceptually, SQL asks: For this customer, does at least one matching order exist? As soon as the answer becomes true, the condition is satisfied. Why This Pattern Matters Many SQL developers initially learn syntax. Over time, they discover that query writing is really about choosing the right mental model. The Presence Pattern encourages you to think in terms of: existence relationshi