标签：#data

Many engineers believe Common Table Expressions (CTEs) are always faster than subqueries. In modern SQL Server (and PostgreSQL), that is a myth . Here is what actually happens under the hood: 1. Inlining & The Query Optimizer By default, the SQL optimizer treats standard CTEs and derived tables (subqueries) almost identically: The engine expands both into the same relational tree. They generate the exact same execution plan and I/O cost . -- Pattern A: Derived Table (Subquery) SELECT DeptID , EmpName , Salary FROM ( SELECT DeptID , EmpName , Salary , DENSE_RANK () OVER ( PARTITION BY DeptID ORDER BY Salary DESC ) AS rnk FROM Employees ) RankedData WHERE rnk <= 2 ; -- Pattern B: Common Table Expression (CTE) WITH RankedData AS ( SELECT DeptID , EmpName , Salary , DENSE_RANK () OVER ( PARTITION BY DeptID ORDER BY Salary DESC ) AS rnk FROM Employees ) SELECT DeptID , EmpName , Salary FROM RankedData WHERE rnk <= 2 ; 2. When CTEs Truly Win: Readability & Pipeline Stacking: You can chain 5 CTEs sequentially without deeply nested pyramid brackets. In-Place Deduplication: In SQL Server, you can run DELETE directly on a CTE, and it deletes duplicate rows straight from the real underlying table! WITH DuplicateCleaner AS ( SELECT CustomerID , Email , ROW_NUMBER () OVER ( PARTITION BY Email ORDER BY RegistrationDate ASC ) AS rn FROM Customers WHERE Email IS NOT NULL ) DELETE FROM DuplicateCleaner WHERE rn > 1 ; -- ✅ Clean in-place deletion! 3. The Big Trap (Spooling Overhead): If you reference the same CTE multiple times in a query (e.g. CTE_A JOIN CTE_A ), SQL Server may execute the underlying CTE query multiple times or create a Lazy Spool in tempdb . -> Fix: For heavy multi-million row reuse, use a Temporary Table ( #TempTable ) with an explicit Clustered Index instead! 💡 How do you choose between CTEs, Temp Tables, and Subqueries in your pipelines? 💼 Connect on LinkedIn: linkedin.com/in/arpitmbangre

What Is Precision Tracking Radar? Precision tracking radar is an active radar sensing system designed to repeatedly measure a selected target and maintain an updated estimate of its state over time. For developers, the important distinction is that precision tracking is not simply repeated target detection. Detection answers: Is there evidence of a target in the current radar measurements? Tracking answers: Does this new measurement belong to an existing target, and how should that target state be updated? A practical precision tracking pipeline can be represented as: RF sensing → target measurement → detection → association → state update → continuous track → mission output That makes precision tracking radar a real-time data-processing system as much as an RF sensing system. A Practical Definition Precision tracking radar is a radar capability that combines repeated target measurements across time to maintain a continuous estimate of target position, motion or other relevant state information. The key word is continuous. A detector can operate independently on each radar update. A tracker has memory. It maintains information from previous measurements and decides how new observations relate to that history. From a software architecture perspective, tracking introduces persistent state into the sensing pipeline. Detection and Tracking Should Be Separate Services A useful radar architecture keeps target detection and target tracking logically separate. The detector processes current radar measurements. The tracker consumes target-related measurements over time. Conceptually: Radar measurement ↓ Detection ↓ Measurement object ↓ Association ↓ Track update ↓ Track state This separation helps developers understand where errors originate. If the detector produces unstable measurements, the tracker cannot fully repair them. If detections are stable but tracks switch between targets, the problem may exist in association. If sensor-relative detections are correct but missio

AI agents are becoming first-class citizens in developer workflows. They can read code, run tests, and deploy apps. But one thing they struggle with is understanding database schemas. Database design tools haven't changed in 20 years. You either use a heavyweight desktop app (Navicat, PDManer) or a pretty but closed web app (dbdiagram). Neither supports versioning, real-time collaboration, or AI agent integration. I built ERD Online to solve this. It's an open-source database design tool that combines Git-like versioning with Figma-like collaboration, plus MCP integration for AI agents. In this article, I'll show you how to let Cursor, Claude, or Cline read and write your database schema through MCP, while you keep full control. Database schema changes are hard to track: Who changed what? When did they change it? Why did they change it? How do I rollback? And now with AI agents, there's a new problem: how do you let an AI agent suggest schema changes without giving it a black box that generates random ER diagrams? The wrong approach: ask AI to "generate an ER diagram for an e-commerce app." You get a diagram, but it has no connection to your actual project, no versioning, and no approval flow. The right approach: let the AI agent read your existing schema, suggest changes, and submit them as a version that you review and approve. That's what ERD Online + MCP does. MCP (Model Context Protocol) is a protocol for AI agents to interact with external tools. Think of it as a USB-C port for AI applications. It standardizes how agents discover and call tools. MCP has three main primitives: Tools : Functions the AI can call (like list_projects or create_version ) Resources : Data the AI can read (like project.json ) Prompts : Pre-defined templates for common tasks ERD Online exposes MCP tools that let AI agents: list_projects : List all your ERD projects get_project : Get a project's projectJSON create_version : Suggest a new version of your schema The key boundary: AI agent

California residents have a legal right to access the data that companies collect about them. Actually exercising that right is a burdensome nightmare.

If you run ClickHouse in production, you're probably on 26.3 LTS. And now 26.8 LTS has been announced, which means the LTS-to-LTS upgrade conversation starts again. Here's the thing most release posts skip: this is not a one-release hop. Going from 26.3 LTS to 26.8 LTS means crossing 26.4, 26.5, 26.6 and 26.7 as well. Every breaking change in those four releases applies to you, and some of the ones most likely to ruin your day aren't in 26.8 at all. So instead of writing another "here are the 26.8 features" post, I wanted to write the thing I'd actually want before scheduling this upgrade: what breaks, what silently changes, what order to do things in, and what you get for the trouble. A note on release timing As of writing (27 August 2026), 26.8 has been announced but is not fully released yet. The release branch is cut and versioned (v26.8.1.1-lts), but the tag and Docker images have not been published yet, and the upstream changelog still marks the 26.8 section as in progress. By the time you read this, the tag has probably landed. Check for yourself: curl -s https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/utils/list-versions/version_date.tsv \ | awk -F '\t' '$1 ~ /^v26\.8\./ {print "26.8 is released - newest: " $1 " (" $2 ")"; f=1; exit} END {if (!f) print "26.8 not released yet"}' version_date.tsv is the list ClickHouse maintains of every released version and its date, so this is the most direct answer available - no auth, no rate limit, nothing to download. As of writing it prints 26.8 not released yet . Worth knowing: the Docker image will lag whatever that command tells you. The Docker Official Images repo trails the GitHub tags by a few patch versions - clickhouse:lts currently resolves to 26.3.20.7 even though 26.3.24.4 has already shipped. So don't treat a missing image as evidence the release hasn't happened. Either way, the timing works in your favour. Historically ClickHouse LTS releases pick up several patch releases quickly - 26.7 had

Most "is this plugin safe?" advice is vibes. I wanted numbers, so I built a dataset. Here's what it found, and exactly how, so you can check my work or build your own. The finding first Of 8,010 WordPress plugins with a publicly documented vulnerability since 2023 (15,534 vulnerability records in total): 3,780 have been removed from the wordpress.org plugin directory. Removal stops updates but doesn't uninstall — affected sites keep running the code. 277 carried a critical (CVSS ≥ 9.0) flaw on record before removal. 2,115 are still installable today with a known vuln and no update in 12+ months — roughly 6.7M active installs combined. The part that surprised me most: "removed from the directory" is nearly invisible to a site owner. No dashboard warning, no email. The plugin just quietly stops getting fixes while sitting on the site. How I built it (no paid APIs) The whole thing runs on two public sources and no API keys. 1. Vulnerability data — the GitHub Advisory Database. It mirrors CVE records including the Patchstack and Wordfence CNA assignments that cover almost all WordPress plugin CVEs. It's a git repo, so a shallow, sparse clone of the advisories/unreviewed/{year} folders gets you the raw JSON: git clone --depth 1 --filter = blob:none --sparse \ https://github.com/github/advisory-database.git Each advisory carries the CVE ID, a CVSS vector string, CWE IDs, and reference URLs. The plugin slug isn't a first-class field — you recover it from the Patchstack/Wordfence reference URLs with a couple of regexes. That alone attributes the large majority of WordPress advisories to a specific plugin. 2. Maintenance signals — the wordpress.org plugin API. For each slug: https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&request[slug]=SLUG That gives install count, last-updated date, tested-up-to version, and support-thread resolution ratio. A 404 (or an {error} body) means the plugin isn't in the directory — but that's ambiguous: it could be removed ,

Most football data scrapers on the market only extract high-level final scores (e.g. 2-1 ). But quantitative sports analysts, data scientists, and predictive betting modelers need granular data: Expected Goals (xG) , Official Referee Assignments , Goal Scorers paired with Assist Providers , and Half-Time vs Full-Time (1H/2H) statistical breakdowns . When I set out to build a professional-grade Flashscore scraper on Apify, I ran into two major engineering challenges: The Memory Problem : Keeping Puppeteer running to scrape hundreds of historical matches consumes over 1.5GB of RAM per run. The Protocol Problem : Flashscore serves its deep statistical feeds using a proprietary pipe-delimited data format ( ~ , ¬ , ÷ ) over CDN endpoints, rather than standard REST APIs. In this tutorial, I'll explain how I engineered the Flashscore Elite Statistics Extractor , how the hybrid Browser + HTTP/2 streaming pipeline drops RAM footprint from 1.5GB to 70MB , how to parse Flashscore's custom feed protocol, and how to pipe the resulting datasets directly into Python and Pandas. 🏛️ The Hybrid Pipeline Architecture To achieve zero proxy reliance for standard runs and ultra-low compute costs, the Actor splits execution into a 2-Phase Hybrid Pipeline : [ League & Season Selection ] │ ▼ ┌───────────────────────────────────────────┐ │ Phase 1: Browser Handshake (Puppeteer) │ │ - Captures x-fsign security tokens │ │ - Extracts countryId & tourId │ └─────────────────────┬─────────────────────┘ │ [ Immediate Browser Shutdown ] (RAM drops from 1.2GB -> 70MB) │ ▼ ┌───────────────────────────────────────────┐ │ Phase 2: Parallel HTTP/2 Feed Workers │ │ - got-scraping with JA3 TLS matching │ │ - Decodes df_st_1_ (Stats) & df_sui_1_ │ └─────────────────────┬─────────────────────┘ │ ▼ ┌───────────────────────────────────────────┐ │ Self-Healing Recovery Pass │ │ - Auto-retries skipped/failed matches │ └─────────────────────┬─────────────────────┘ │ ▼ ┌───────────────────────────────────────────┐

Meta has detailed MTIA 300, its first in-house accelerator optimized for training ranking and recommendation models. By Matt Foster

Startups building B2B products reach for multi-tenancy in PostgreSQL the same way on day one: one shared database, one set of tables, and a tenant_id column marking who owns each row. That is the correct call, and it stays correct for a long time. However, when that column is enforced by application code rather than by the database, a single forgotten predicate stops being a bug and becomes a disclosure event, and a disclosure event is one of the very few engineering failures that lands straight on your balance sheet as stalled enterprise deals, an unplanned legal bill, and a security review you can no longer pass. By understanding what multi-tenancy actually guarantees, which isolation model fits your stage, and how Row-Level Security moves that guarantee out of your codebase, startup CTOs and Fractional CTOs can make the tenant boundary hold without slowing the team down. (If you want to skip the theory, jump straight to the connection pooler trap that switches Row-Level Security off in production, what it costs in query performance, or when it is genuinely time to leave the shared schema.) Because "enforced by application code" means something very specific in practice. It means a promise that everyone will remember to filter on tenant_id , and that promise is the single most expensive line of undocumented policy in your entire codebase, because it holds perfectly for about fourteen months, right up until the afternoon a tired engineer ships a reporting endpoint that joins four tables and forgets the predicate on exactly one of them, and then a customer opens a dashboard and sees somebody else's invoices. That is not a bug. A bug is something you fix on Monday. A cross-tenant data leak is a disclosure event, which means legal gets involved, your enterprise prospects get an email from their own security team, and the deal that was supposed to close your Series A quietly moves to next quarter and then to never. The uncomfortable part is that this is not a story abo

Classic Machine Learning Through the Eyes of an SRE — Part 9: Isolation Forest The algorithm in one line: Isolation Forest scores how anomalous a point is by how few random cuts it takes to separate that point from everything else. No model of normal, no loss function, nothing optimized. ← Previous: Part 8 — Hierarchical Clustering Fails Beautifully · Next: this is the series finale — start at Part 1 . Every anomaly detector I had studied models what NORMAL looks like, then calls the leftovers outliers. K-Means: far from every centroid. DBSCAN: in the noise bucket. Sensible, and intuitive. Isolation Forest does not bother. It never models normal at all. It goes straight at the rare points with a single question: how few random cuts does it take to isolate you? Random cuts, literally. Pick a feature at random, pick a split value at random between that feature's min and max, repeat. A point that separates from the crowd in three cuts is anomalous. A point buried in the middle of a dense mass takes thirty. Grow hundreds of these random trees, average the isolation depth for each point, and you get an anomaly score. There is no loss function here. No optimization, not even the local kind that decision trees do at every split. Every cut is a coin flip, and the power comes entirely from averaging, which is the forest trick from the supervised half of this series now applied to pure randomness. Cheap randomness plus averaging beats careful modeling, as long as the target is something randomness naturally exposes. Rarity is exactly that. Sometimes the winning move is to optimize less. That sentence would have gotten me laughed out of my first ML study session. It is also this finale's thesis. The part I had completely backwards Here is the thing I did not know until I read the original paper properly, and it is the opposite of every instinct a decade of ops gave me. Isolation Forest deliberately trains each tree on a small subsample of your data, and this is not a performan

At TechCrunch Disrupt 2026, the AI Stage is back to dig into the single hottest topic in the community for the past few years, presented by Google for Startups.

Hoomanely has developed a smart bowl to measure and record dogs' feeding data, then tells owners if behaviors change.

After a 10-week hiatus since the last batch of Spring ecosystem releases, there was a flurry of activity during the week of August 17th, 2026, highlighting first milestone releases of: Spring Boot, Spring Framework, Spring Data, Spring Security, Spring Integration, Spring HATEOAS, Spring Modulith, Spring Batch, Spring AMQP and Spring for Apache Kafka. By Michael Redlich

The lakehouse projects spent this week arguing about boundaries. Iceberg decided where conformance testing lives and started sketching the REST API shape that V4 tables will need. Polaris argued about what a committer owes a project when LLMs make pull requests cheap. Parquet pulled a feature apart because two proposals were reaching for the same mechanism. DataFusion and Iceberg Rust opened a joint thread about which repository should own their integration. Every one of those debates is a question about ownership, and the answers this week tell you a lot about how these communities plan to scale. Apache Iceberg The single biggest outcome of the week was the creation of a new repository. Neelesh Salian, working with Sung Yun and Andrei Tserakhau, called a vote to create apache/iceberg-verification , a standalone home for language-neutral conformance fixtures that every Iceberg implementation can run against. The vote passed with five binding +1s from Russell Spitzer, Sung Yun, Matt Topol, Daniel Weeks, and Amogh Jahagirdar, plus twenty-two non-binding votes. That is a wide turnout. The names on the non-binding list read like a roll call of the Rust, Python, Go, and Java maintainers, which is the point. Salian will now work with a PMC member to stand the repository up. The reason this matters goes beyond tidiness. Iceberg has at least five serious implementations today across Java, Python, Rust, Go, and C++. Each one carries its own test fixtures and its own understanding of edge cases in the spec. When two implementations disagree about how to interpret a manifest list, users find out the hard way. A shared set of fixtures that every implementation reads from one place turns spec ambiguity into a failing test rather than a production surprise. The 29 messages in the vote thread also included a fair amount of discussion about what belongs in the first batch of fixtures, and the conversation is worth reading if you maintain a client. The second major thread was about

AI’s water footprint is growing, but location and cooling technology make a difference.

Most LlamaIndex setups end up with two separate backends once you go beyond plain vector search: a vector store for VectorStoreIndex , and a separate graph database for PropertyGraphIndex when you need relationship-aware retrieval (GraphRAG). Two services, two connection strings, two things to keep in sync. This is a walkthrough of backing both index types with SynapCores instead — one engine, one connection, both index types. Setup docker run -d --name synapcores -p 8080:8080 \ -e AIDB_ACCEPT_LICENSE = 1 \ -v synapcores-data:/var/lib/synapcores \ ghcr.io/synapcores/community:latest pip install llama-index llama-index-vector-stores-synapcores llama-index-graph-stores-synapcores Both integration packages are independently published on PyPI: llama-index-vector-stores-synapcores llama-index-graph-stores-synapcores Vector store — standard RAG from llama_index.core import VectorStoreIndex , StorageContext , Document from llama_index.vector_stores.synapcores import SynapCoresVectorStore vector_store = SynapCoresVectorStore ( uri = " http://localhost:8080 " , embedding_dim = 1536 ) storage_context = StorageContext . from_defaults ( vector_store = vector_store ) docs = [ Document ( text = " SynapCores runs vector search, graph traversal, and SQL in one engine. " )] index = VectorStoreIndex . from_documents ( docs , storage_context = storage_context ) query_engine = index . as_query_engine () response = query_engine . query ( " What does SynapCores combine into one engine? " ) print ( response ) The vector store implements the full BasePydanticVectorStore ABC — add , delete , query , delete_nodes , clear , plus the async surface. Metadata filtering supports the full MetadataFilters grammar: all 12 operators ( EQ , NE , GT / GTE / LT / LTE , IN , NIN , TEXT_MATCH , TEXT_MATCH_INSENSITIVE , CONTAINS , IS_EMPTY ) with AND / OR / NOT and nested groups — so you're not giving up filtering power by moving off a dedicated vector DB. If you already have data in SynapCores from a prev

The UK’s energy regulator is using a variety of tricks to keep speculative data center projects from plugging into the power grid. The country’s AI ambitions hang in the balance.

Short answer: use a scheduled cleanup endpoint when one indexed, bounded pass can finish predictably; use a queue when cleanup must be divided into independently retriable batches. For a fintech SaaS that fans out shipment updates to many subscribers, latency and cost should be judged at the system boundary: a cheap cleanup run is not a good bargain if it contends with delivery or leaves retention evidence incomplete. The first design decision is to keep shipment fan-out separate from retention work. A shipment update has a latency-sensitive path. Expired subscriptions, old delivery attempts, and temporary fan-out records usually have a policy-driven path. They may share a database, but they should not share an unbounded transaction or an execution budget. This distinction matters more than the spelling of a cron expression. It also gives the team a useful test: can the cleanup be repeated safely while the shipment update path continues to make progress? How should a Node.js SaaS choose a cron or queue for scheduled cleanup? Measure the worst case first. Count eligible records by tenant, check the relevant index, estimate lock pressure, and measure a bounded pass while the database is serving normal shipment traffic. The median duration is not the decision variable; the tail is. A scheduled data cleanup is a good fit for one HTTP-triggered run when its cutoff, tenant scope, batch size, and completion state can be recorded and the run has room to finish before its execution limit. The cutoff should be computed by the application and persisted with the run. A schedule has jitter, and a paused schedule may not replay every missed invocation. “Delete records older than the cutoff captured at run start” is therefore more auditable than silently recalculating the boundary for every page. The query should also exclude legal holds, active disputes, and any retention exception required by the business policy. Keep it bounded. The boundary is operational. When a tenant can mo

Today's two tasks are both about a new base. A feature branch that needs to sit on top of a master that has moved. A database instance that needs to come back from a snapshot taken when things were fine. In each case, the interesting question is the same: what carries over, and what do you have to say out loud? One Git task, one AWS task. Rebase a feature branch onto master without creating a merge commit, then snapshot an RDS instance and restore it into a new one. The tasks come from the KodeKloud Engineer platform. Rebase: not moving commits, replaying them The requirement was specific, and the specificity is the lesson. A developer's feature branch was behind master. Bring it up to date without losing any feature work, and without a merge commit. That second clause rules out git merge master . Merge joins two histories and records the join, which is the merge commit. Rebase does something else entirely. cd /usr/src/kodekloudrepos/media git branch git log --oneline --graph --all --decorate git checkout feature git rebase master git log --oneline --graph --decorate Git's own documentation describes what happens under git rebase master : it lists the commits on your branch that are not on master, checks out master, and then replays each of your commits on top of it, one at a time, in a way it compares to running git cherry-pick for each one. Replays. Not moves. Every commit that comes out the other side has a new hash, because a commit's identity includes its parent, and the parent is different now. Your work is preserved, the commits carrying it are not the same objects they were. That is exactly why there is no merge commit. Rebase does not join two histories, it rewrites yours so it looks like it was always based on master's current tip. You get a straight line, at the cost of a history that is no longer a record of what actually happened. Two things I had to be deliberate about. Direction. Rebase applies to the branch you are standing on and takes the branch yo

More than 15 politicians from across the country have signed on to the AI Pact, vowing to regulate data centers and AI. “We’ve got to get this right,” says Senate candidate Dan Osborn of Nebraska.