标签：#data

Most cloud systems make one authorization decision at login. Everything after runs on trust established at authentication time. For systems handling regulated data, that gap is where breaches happen. This article presents a continuous authorization architecture covering risk-tiered evaluation, behavioral baselines, privacy-preserving audit trails, and a phased and incremental rollout. By Venkata Nedunoori

In a new Windows Developer Blog post titled "Windows platform security for AI agents", Microsoft positions Windows as the trustworthy operating system for autonomous agents and introduces the Microsoft Execution Containers (MXC) SDK as the core of that strategy. The post argues that containment, identity and manageability must be built into the operating system. By Matt Saunders

ClickHouse HTTP API: A Complete Beginner's Guide Introduction When most people think about interacting with a database, they usually imagine connecting through a database client or application. However, ClickHouse also provides a simple and powerful HTTP API that allows you to query and manage your database using standard HTTP requests. The ClickHouse HTTP API provides a universal interface for communicating with your ClickHouse server. Since almost every programming language and automation tool supports HTTP, it becomes an excellent choice for integrations, monitoring, scripting, and lightweight applications. In this guide, you'll learn what the ClickHouse HTTP API is, why it's useful, and how to perform common database operations using simple HTTP requests. What Is the ClickHouse HTTP API? The ClickHouse HTTP API is a built-in interface that enables clients to communicate with a ClickHouse server using the HTTP protocol. Instead of connecting through the native TCP protocol, you simply send HTTP requests and receive responses in formats such as JSON, CSV, TSV, XML, or plain text. The HTTP interface is: Language agnostic Easy to integrate with web applications Firewall friendly Simple to test using tools like cURL, Postman, or a web browser Because of its simplicity, the HTTP API is widely used for automation, dashboards, data pipelines, and monitoring systems. Why Use the HTTP API? The ClickHouse HTTP API offers several advantages: No dedicated database driver is required. Works with virtually every programming language. Easy integration with REST-based applications. Supports multiple output formats. Ideal for automation and scripting. Perfect for cloud-native applications and microservices. Common Operations Using the HTTP API, you can: Execute SQL queries Insert data Create and modify tables Retrieve query results Export data in different formats Automate database operations Authentication Options ClickHouse supports multiple authentication methods when using th

DuckDB 1.4.5 LTS, pgEdge ColdFront Beta, and SQLite's FCNTL_PDB Internals Today's Highlights This week's highlights feature the latest DuckDB 1.4.5 LTS release, a new open-source beta for PostgreSQL data tiering, and a deep dive into an obscure SQLite internal file control operation. These updates offer performance, architectural flexibility, and internal insights across the SQLite ecosystem. Announcing DuckDB 1.4.5 LTS (Andium) (DuckDB Blog) Source: https://duckdb.org/2026/06/17/announcing-duckdb-145.html The latest Long Term Support (LTS) release of DuckDB, version 1.4.5 named "Andium", has been announced, primarily focusing on bugfixes and performance enhancements. DuckDB, an in-process analytical processing database, continues to refine its engine for enhanced stability and efficiency in embedded and edge computing environments. While the announcement is concise, LTS releases are crucial for developers and organizations relying on a stable and well-tested version for their data pipelines and analytical workloads, ensuring long-term compatibility and reliability. This update is vital for maintaining the robustness of applications that utilize DuckDB for local data transformations, complex analytical queries, and other high-performance data operations. Users of previous 1.4.x versions are encouraged to upgrade to benefit from the accumulated stability improvements and minor speedups, all without introducing major breaking changes. This commitment to incremental improvements and stable releases solidifies DuckDB's position as a premier solution for embedded analytical database needs, making it a reliable choice for critical projects. Comment: An LTS release, even with bugfixes, is always welcome from DuckDB. It reinforces their commitment to a stable and performant analytical database that I frequently use for local data processing and reporting. Introducing ColdFront: Seamlessly Uniting OLTP, Analytics and AI Workloads on PostgreSQL (Planet PostgreSQL) Source: htt

FERC told grid operators to give data centers a fast lane for interconnections, but it failed to address electricity supply shortages.

A data breach involving government-issued ID documents affects over three million people in Texas.

The software engineers filed a complaint with Seattle’s civil rights office accusing Amazon of illegally retaliating against them for expressing their personal political beliefs.

Vinay Chella and Akshat Goel discuss the challenges of running traditional CDC across heterogeneous databases during peak order traffic. They explain how Debezium hit limits under high load and share how they built Write-Ahead Intent Log (WAIL) - a custom architecture that utilizes a dumb producer proxy and a smart consumer pattern to cleanly separate the intent from the state payload. By Vinay Chella, Akshat Goel

pgrac is an open attempt to rebuild Oracle RAC's core machinery (shared-everything storage, multiple active nodes all writing one database, a cluster-wide change number) on top of PostgreSQL 16. Build log #1 laid out the four problems that fight back. This one is about the problem that turns a node failure into silent data corruption, and the first, deliberately modest, layer pgrac ships against it. The failure mode In a shared-nothing cluster an evicted node is mostly harmless: it owns its own disks, so the cluster routes around it. In a shared-everything cluster the same event is dangerous, because every node writes the same storage. Picture the classic split: node 2 misses heartbeats, the cluster declares it dead and remasters its work elsewhere, but node 2 is not actually dead. It is frozen on a long GC pause, or its interconnect NIC flaked, and it is about to wake up and finish the write it started. Now two nodes believe they own the same blocks, and shared storage will accept both writes. That is not a crash. It is corruption you find three days later. Oracle RAC's answer is I/O fencing: before remastering a dead node's resources, you make certain it can no longer touch the storage, with STONITH, SCSI-3 persistent reservations, or a hardware watchdog. The node is fenced at a layer below its own software, because the whole point is that you cannot trust the dead node's software to behave. That hardware layer is real work, and it is not what pgrac built first. What it built first is the layer above it: an in-process cooperative write-fence, now default-ON. The rest of this is precise about what that does and does not buy you, because "we have fencing" is the kind of claim that is worth less than nothing if it is overstated. A fence needs an authority everyone can agree on You cannot fence on local opinion, because the whole problem is that the dead node disagrees about being dead. Authority has to live on durable, shared, quorum-backed storage. pgrac writes a sm

Microsoft recently introduced at Build 2026 Microsoft Scout, an always-on agent. Scout belongs to a new category of agents Microsoft called Autopilots: always-on agents that work autonomously on a user’s behalf with their own identity, without needing to be prompted each time. Microsoft Scout integrates with Work IQ and is based on the open-source agent framework OpenClaw. By Bruno Couriol

A practical walkthrough of SQL-First persistence: no XML, no Mapper interfaces, no generated queries. I maintain a Java backend that handles ~1M requests/day. For persistence, we used to run MyBatis. The XML was manageable at first, then it wasn't. Dynamic conditions became <if> tag soup. A simple join query needed three files and two languages. We switched to a simpler approach. Here's how it works for the most common case: single-table CRUD. What You Need Java 21+ Spring Boot (any 3.x) A database (H2 for the demo, MySQL/PostgreSQL for production) That's it. No XML parser, no code generator, no annotation processor. The Project pom.xml src/main/java/example/ DemoApplication.java user/ User.java -- entity UserDao.java -- data access UserCond.java -- query conditions src/main/resources/ application.yml schema.sql Three Java files for a complete CRUD API. The Entity @Data @Builder @Table ( "sys_user" ) public class User { @Id private Long id ; private String name ; private Integer age ; private String email ; // ... other fields // These four are auto-managed: private LocalDateTime createTime ; private Long createBy ; private LocalDateTime updateTime ; private Long updateBy ; private Byte dr ; // 0 = active, 1 = soft-deleted } @Table maps to the database table. @Id marks the primary key (Snowflake ID by default). The audit fields and soft-delete marker are handled automatically—you don't set them in business code. The DAO @Repository public class UserDao extends BaseDao < User > { // Empty. All CRUD methods inherited. } BaseDao provides save , saveBatch , update , delete , findById , list , page , count , exists . For single-table operations, this is all you need. The Conditions @Getter @Setter @Builder public class UserCond extends BaseCondition { private String name ; private Integer ageMin ; private Integer ageMax ; private Byte dr ; private Object [] ids ; @Override protected void addCondition () { and ( "name LIKE" , name , 3 ); // 3 = %value% and ( "age >=" , ag

PostgreSQL Error 22036: non numeric sql json item PostgreSQL error code 22036 ( non numeric sql json item ) occurs when a SQL/JSON path expression attempts to perform a numeric operation on a JSON item that is not a number — such as a string, boolean, array, or object. This error was introduced alongside the SQL/JSON Path feature in PostgreSQL 12 and typically surfaces in queries using jsonb_path_query , jsonb_path_exists , or the @@ and @? operators. Top 3 Causes 1. Numeric Values Stored as Strings The most common cause is JSON data where numbers are stored as quoted strings (e.g., "price": "100" instead of "price": 100 ). This frequently happens with data from external APIs or legacy systems that don't enforce type consistency. -- Triggers 22036: "price" is a string, not a number SELECT jsonb_path_query ( '{"price": "100"}' , '$.price + 50' ); -- ERROR: non numeric SQL/JSON item -- Fix: Use the .double() conversion method in JSON Path SELECT jsonb_path_query ( '{"price": "100"}' , '$.price.double() + 50' ); -- Result: 150 -- Alternative fix: Cast at the SQL level SELECT ( data ->> 'price' ):: numeric + 50 FROM ( SELECT '{"price": "100"}' :: jsonb AS data ) t ; -- Result: 150 2. Arithmetic Applied Directly to Arrays or Objects Developers sometimes write JSON Path expressions that target an entire array or object instead of individual elements, then attempt arithmetic on the result. -- Triggers 22036: $.scores returns an array, not a number SELECT jsonb_path_query ( '{"scores": [80, 90, 70]}' , '$.scores + 10' ); -- ERROR: non numeric SQL/JSON item -- Fix: Target a specific array index SELECT jsonb_path_query ( '{"scores": [80, 90, 70]}' , '$.scores[0] + 10' ); -- Result: 90 -- Fix: Use wildcard to apply operation to each element SELECT jsonb_path_query ( '{"scores": [80, 90, 70]}' , '$.scores[*] + 10' ); -- Results: 90, 100, 80 -- Fix: Use unnest for aggregation use cases SELECT elem :: numeric + 10 FROM jsonb_array_elements ( '{"scores": [80, 90, 70]}' :: jsonb ->

I recently calibrated a recovery-rate model that had only two weak features. Its point accuracy was almost nothing — R² basically zero. I expected its uncertainty estimates to be junk too. They weren't: the 90% conformal prediction intervals covered ~89% of held-out outcomes. Valid, just wide . That surprised me enough to nail it down, because it contradicts a belief a lot of us carry around: "my model isn't accurate, so I can't trust its uncertainty." For split conformal prediction, that's backwards. Here's the precise statement, a runnable demo, and the one caveat that actually bites. Coverage is a property of the procedure, not the model Split conformal prediction gives a distribution-free, finite-sample marginal coverage guarantee : P( Y ∈ Ĉ(X) ) ≥ 1 − α and it holds for any point model, as long as the calibration and test data are exchangeable. The model is a black box. You fit it however you like, then on a held-out calibration set you take the (1−α) quantile of the absolute residuals, and that quantile becomes the half-width of your intervals. Nowhere does that construction require the model to be good. A bad model just has large residuals, so the calibration quantile is large, so the intervals are wide — wide enough to still cover at the stated rate. Accuracy doesn't buy you validity ; it buys you efficiency (narrower intervals at the same coverage). The demo (numbers are reproducible, seed fixed) Same dataset and target, three models from strong to useless, target coverage 90%: model R² marginal coverage mean interval width gradient boosting 0.741 0.895 5.39 weak linear (1 noisy feature) 0.061 0.905 10.39 predict-the-mean −0.000 0.907 10.83 All three land at ~90% coverage. The only thing that changes is width: the good model's intervals are half as wide . That's the whole story in one table — validity is constant, efficiency tracks accuracy. import numpy as np from sklearn.linear_model import LinearRegression from sklearn.ensemble import GradientBoostingReg

An alleged Russian-speaking group of cybercriminals is reportedly compromising and targeting several major companies that use Fortinet Firewalls and VPNs through previously known passwords.

You type "k8s deployment troubleshooting" into your documentation search. The top result is a page about Kubernetes architecture that never mentions the word "troubleshooting." It is exactly what you need. BM25 would have missed it entirely. This is the promise of vector search: finding documents by meaning, not just matching words. In 2025 and 2026, vector search has moved from niche ML engineering to a core Elasticsearch capability. If you are building search for AI applications - RAG pipelines, semantic Q&A, recommendation systems - understanding how Elasticsearch handles vectors is no longer optional. I have spent the past year building RAG pipelines at Cloudera, and I have learned that vector search is powerful but easy to misuse. This post covers what works, what does not, and how to implement it in production. Why Vector Search Matters (And When It Does Not) BM25, which we covered in a previous post, is brilliant at matching exact terms. But it is fundamentally lexical. It does not understand that: "k8s" and "kubernetes" are the same thing "docker container" and "containerization" are related concepts "out of memory error" and "heap exhaustion" describe the same problem Vector search solves this by converting text into high-dimensional numerical vectors (embeddings) where semantically similar content lives close together in vector space. A query for "k8s deployment troubleshooting" gets embedded into a vector, and Elasticsearch finds the nearest document vectors - even if they do not share a single keyword. But vector search is not a replacement for BM25. It is a complement. BM25 is faster, requires no ML infrastructure, and excels at exact-term matching. Vector search is slower, requires embedding models, and shines at conceptual similarity. The best search systems in 2026 use both. How Elasticsearch Stores and Indexes Vectors Elasticsearch introduced the dense_vector field type in version 7.x and has dramatically improved it through 8.x and into 2026. Here

Why Your Chinese-Tenured Faculty Resume Won’t Work in US Industry US data scientist hiring managers scan a resume in 7–15 seconds looking for one thing: evidence you can solve business problems with data. A Chinese faculty resume often leads with tenure status, publication counts, and grant amounts—none of which translate to industry value. Worse, the CV-style length and Chinese-specific qualifications (e.g., “Professor of Record,” “National Natural Science Foundation PI”) confuse HR software and recruiters unfamiliar with that system. You need to strip the academic frame and rebuild around what a US data scientist does: clean messy data, build predictive models, deploy to production, and communicate results to non-technical stakeholders. Think of every faculty achievement as raw material you must reframe. Core Rewriting Rules: From Academic to Industry Rule 1: Replace Tenure Rank with a US-Equivalent Data Science Title Do not list “Tenured Associate Professor” unless it is your most recent position at a well-known university (e.g., Peking University, Tsinghua). Instead, use a title that reveals your function: “Senior Data Scientist – Research Computing” or “Lead Data Scientist – Machine Learning Research Lab.” The point is to signal the job function, not the academic rank. Example: Before: “Tenured Associate Professor, School of Computer Science, Fudan University” After: “Senior Data Scientist / Research Lead, Fudan University AI Lab” Rule 2: Translate Every Accomplishment into a Business-Relevant Metric Chinese faculty resumes often say “published 15 papers in top-tier journals” or “secured ¥3M in research funding.” That means nothing to a hiring manager at a fintech startup. Instead, describe what you did with the data and the outcome. Concrete example – before and after: BEFORE (faculty bullet): “Led research project on deep learning for medical image segmentation; published 3 papers in IEEE TMI.” AFTER (industry data scientist bullet): “Built and validated a co

TL;DR — UUID v7 trie 13× plus vite que v4 en simulation B-tree (1M entrées), expose une empreinte mémoire identique, mais révèle son timestamp d'émission. UUID v4 reste le choix "zéro réflexion" pour les identifiants isolés. Le reste de cet article vous donnera les données pour décider. Introduction Les UUIDs sont omniprésents dans les systèmes modernes : clés primaires de bases de données, identifiants de sessions, tokens de traçabilité. Pourtant, le choix de la version impacte directement les performances en écriture et en lecture, la fragmentation des index, et — dans certains contextes — la confidentialité des données. UUID v4 (RFC 4122, 2005) est aujourd'hui la version par défaut de presque tous les ORM et frameworks. UUID v7 (RFC 9562, 2024) est son successeur moderne, conçu pour corriger son principal défaut : le désordre lexicographique. Dans cet article, nous allons mettre les deux en face avec des benchmarks réels sur des volumes de 100 000 à 10 millions d'UUIDs , analyser leur structure bit par bit, et vous donner une grille de décision claire. Structure interne : ce que contiennent ces 128 bits UUID v4 xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx Bits Contenu 0–47 Aléatoire 48–51 Version (0100 = 4) 52–63 Aléatoire 64–65 Variant (10) 66–127 Aléatoire 122 bits d'entropie pure. Aucune information temporelle. Chaque UUID est statistiquement indépendant des autres. UUID v7 019ed5c8-2a2f-7974-91f2-6ba1f313dcfa └──────────────┘ 48 bits = timestamp Unix en millisecondes Bits Contenu 0–47 Timestamp Unix (ms) 48–51 Version (0111 = 7) 52–63 Aléatoire (sub-ms ou compteur) 64–65 Variant (10) 66–127 Aléatoire 74 bits d'entropie + 48 bits de temps. Naturellement monotone : deux UUIDs générés dans la même milliseconde sont toujours distincts et ordonnés de façon cohérente. Lecture du timestamp (Python) : import uuid6 u = uuid6 . uuid7 () b = u . bytes ts_ms = int . from_bytes ( b [: 6 ], ' big ' ) # → 1781703125553 (ms depuis epoch Unix) Depuis la sortie de nos benchmarks : [00

Aditya Kumarakrishnan explains how to move past the "amnesia phase" of AI. He shares a blueprint for engineering leaders to build modular agent frameworks using CoALA, leverage decades of process science for scalable workflows, and "terraform" legacy environments into robust, event-sourced artifacts capable of handling unpredictable, cross-functional agent demands. By Aditya Kumarakrishnan

GitHub has introduced the GitHub Copilot app, a desktop control centre for agent-native development that aims to keep engineers in charge while AI agents handle more coding work. Mario Rodriguez writes on the GitHub blog that the recent wave of coding agents has brought faster delivery but also "disjointed workflows, more context switching, and too much time spent reviewing agent-generated code". By Matt Saunders

Time-series data is one of the most common types of data generated by modern applications. Every log entry, API request, metric, transaction, sensor reading, or user interaction is recorded with a timestamp, making time the primary dimension for analysis. As organizations collect billions of these records, efficiently storing and querying them becomes increasingly challenging. This is where ClickHouse® excels. Although ClickHouse is not a dedicated time-series database, its columnar storage architecture, vectorized query execution, high compression ratios, and massively parallel processing make it an excellent choice for time-series analytics at scale. It is capable of ingesting large volumes of data while delivering analytical queries in milliseconds. The article begins by explaining the fundamentals of time-series data and highlighting common real-world use cases such as application monitoring, IoT sensor data, financial market analysis, server metrics, user activity tracking, and business analytics. These workloads typically involve continuous data ingestion, time-based filtering, aggregations, and trend analysis. One of ClickHouse's biggest strengths is its optimization for analytical workloads. Since data is stored column-wise rather than row-wise, only the required columns are read during query execution. Combined with compression and vectorized processing, this significantly reduces I/O and improves query performance over massive datasets. The article also demonstrates how to create an optimized table for time-series workloads using the MergeTree engine. Proper partitioning by month and ordering data by dimensions and timestamps help ClickHouse prune unnecessary partitions and efficiently locate relevant data during queries. Several practical SQL examples are covered, including: Filtering records within a specific time range Aggregating metrics by hour, day, week, or month Calculating averages, sums, minimums, and maximums Grouping events over time Working wi