标签：#Data

1. Introduction Excel is much more than a spreadsheet for entering numbers. It can be used as a data-analysis tool that helps analysts inspect, validate, filter, summarize, and prepare raw data before deeper analysis begins. In typical analytics, the quality of the final work depends heavily on the quality of the data used; therefore, data cleaning is not an optional step—it is the foundation of effective data analysis. This article demonstrates key Week 1 Excel concepts _using an employee dataset containing _employee IDs, names, departments, gender, marital status, hire dates, salaries, educational level, performance score among others. The raw file intentionally contains common data-quality issues: inconsistent capitalization on the First and Last names, blank records, duplicate employee records, varying department names, currency and dates that need review. By working through these issues, the article shows how Excel’s formatting tools, text functions, filters, conditional formatting, numerical functions, conditional summaries, and date functions can turn a messy workbook into an analysis-ready dataset. 2. Why Data Cleaning Matters Data cleaning is more than just about removing errors. By standardizing formats and categories, we make datasets more transparent, usable, and valuable for management analysis and reporting purposes. Data analysis is simple – garbage in, garbage out. A dashboard or prediction can appear professional, but can be misleading if the underlying data has duplicates, blank values, inconsistent categories or incorrectly formatted text and dates. For example, “IT” “I.T.” and “Information Tech” can be viewed as different department values if naming is not standardized. Duplication of an employee ID can inflate employee counts and department totals. A blank performance score might mean that something is missing and should be looked into and dates saved as text cannot be reliably used in calculations such as employee tenure checks. A good practice

Sometimes, a change that looks simple on the surface is not actually that simple. Imagine that you need to replace the source table feeding a refined or trusted table in a data pipeline. At first, it might look like a one-line change: update the table name, deploy the code, and move on. But in a real data platform, there is usually much more behind that change. There are dependencies, scheduling rules, upstream and downstream processes, resource consumption, concurrency, data lineage, and, sometimes, assumptions that were not immediately obvious when the pipeline was first created. I recently had to look into exactly this kind of situation in an Apache Airflow project, and one of the questions that came up was: Should this DAG be scheduled using a cron expression, or should it be triggered based on an Asset? The answer, as usual in software engineering, is: it depends. And understanding why it depends is much more important than simply knowing how to configure either option. Cron: the familiar way of scheduling a DAG Let's start with the simplest and most familiar option: a time-based schedule. With Airflow, we can define a DAG to run according to a cron expression: with DAG ( dag_id = " my_pipeline " , schedule = " 0 13 * * 0 " , catchup = False , ): ... In this example, the DAG is scheduled to run every Sunday at 1 PM. In a real environment, we might have different schedules for different environments. For example: Environment Schedule Development Saturday at 1 PM Homologation Sunday at 1 PM Production Monday–Friday at 1 PM The important characteristic here is that the schedule is based on time . If the DAG is configured to run at 1 PM every Sunday, Airflow will try to run it at that time, regardless of whether the data it depends on has actually changed. This is not necessarily a bad thing. In fact, sometimes this is exactly what we want. But there is another approach. When data becomes part of the schedule Modern data pipelines often have dependencies that are b

The second you hit Enter Friday night. You ask Cursor for a query: join orders to users, sort by last login. Three seconds later, an answer arrives with DBA-level confidence: SELECT o . id , o . amount , u . last_login_at FROM biz_order o JOIN sys_user u ON u . id = o . user_id ORDER BY u . last_login_at DESC ; Paste it into your client. Enter: ERROR: column "last_login_at" does not exist LINE 2: SELECT o.id, o.amount, u.last_login_at There is no last_login_at column. There never was. The model did not know — it just decided the column "should" exist. This failure has a name: invented column This is not "AI is not smart enough yet." It has a name — invented column : the model fabricates a plausible column name with no factual source, then writes it into a JOIN with unshakable tone. Invented columns are dangerous because they do not look like errors . last_login_at appears on 90% of user tables. Syntax is correct. Naming is conventional. Indentation is perfect. Mixed into ten correct JOINs, you will not catch it line by line. You find out in code review — or worse, in production logs. Three things you already tried A better prompt. "Do not invent column names; only use the schema I provide" — added to the system prompt. Works day one. By day three, long context and the model forgets. A prompt is a wish, not a constraint. @schema.sql . Export DDL and drop it into context. The most honest approach today — but two holes: it goes stale (last week's export does not know this week's column), and nobody maintains it (not in any approval flow; anyone can edit it; drift from the real database goes unnoticed). Live catalog MCP. Let the Agent query information_schema directly. Directionally correct — give the model a fact source instead of guesses. Tools like postgres-mcp and cloud vendor MCPs do solve half of "stop hallucinating column names." Worth acknowledging. Live catalog only gets you halfway Wire production into the IDE and you hit four walls: Permission-filtered inform

By Michael Nocito , data analyst · Published August 8, 2026 By the end of this page you will be running SQL directly against a CSV file on your machine, with no import step, no CREATE TABLE , and no schema written by hand. DuckDB reads the file where it lies, works out the column types itself, and gives you a normal SQL result. It takes one command to install and about a minute to prove. Here is what to actually do today. Run python -m pip install duckdb , then write a query with your CSV's filename in quotes where the table name would normally go. That is the entire idea, and everything else on this page is a consequence of it. The short version: a file is a table. It suits large files and folders of files, it does not replace SQLite for a shared database you keep, and section 6 says which to use when. The missing import step is the one idea worth the page, so it gets the picture. The original carries a diagram here. In words: Two horizontal sequences. The upper sequence runs through four stages joined by arrows: a file icon, then a box representing a schema being written, then a database cylinder, then a result grid. The lower sequence has only two stages joined by a single long arrow: the same file icon on the left and the same result grid on the right, with the middle two stages absent and the empty space where they used to be left visibly blank. Every output on this page is real. Run on 8 August 2026 with DuckDB 1.5.5 on Windows, against a 412-row CSV exported from the Chinook sample database. The numbers match the ones in the sample-database guide and the Python guide on purpose, because it is the same data through three different tools. 1. Install it Before the explanation: every database you have met so far needed you to create a table before you could put anything in it. What would have to be true for that step to be unnecessary? python -m pip install duckdb That is the whole installation. No server, no service running in the background, no configuration fi

By Michael Nocito , data analyst · Published August 8, 2026 By the end of this page you can load a CSV into pandas, find out in twenty seconds what type every column became, stop the identifier columns losing their leading zeros, get dates read the way they were written, and turn a money column that arrived as text into numbers. It is about twenty-five minutes, and every output below was produced by running the code. Here is what to do today, the moment after you first load a file. Run df.dtypes . Not df.head() , which shows you what the values look like, but dtypes , which shows you what they are. A column of identifiers that says int64 has already lost its leading zeros, and a money column that says object or str is text that will refuse to add up. The short version: read_csv reads characters and guesses a type per column. The guess is usually right, it is silent when it is wrong, and four arguments replace guessing with instruction. The same characters becoming two different values is the idea, so it gets the picture. The original carries a diagram here. In words: On the left, a strip of five small square boxes holds one character each, reading zero, eight, zero, five, three, as the characters appear in the file. Two arrows branch out from that strip. The upper arrow leads to a strip of five boxes in which the first box is empty, crossed through and outlined in amber, while the remaining four hold eight, zero, five and three; the leading character has been discarded. The lower arrow leads to a strip of five boxes holding zero, eight, zero, five and three, identical to the original, outlined in blue. Both destinations came from the same source strip, and only one of them still contains everything the file did. Every output on this page is real. Run on pandas 3.0.2 against a small CSV built to contain the four problems every real export has: an identifier with leading zeros, ambiguous dates, a text marker for missing values, and money with a thousands separator. If

About a year ago, I was building a terminal-based workflow manager called Glyph.Flow. It was mostly a learning project. I wanted to understand Python better, experiment with Textual, think about commands, state, configuration, logging, and all the small architectural decisions that suddenly appear when a script stops being a script. Somewhere between then and now, the workflows became a little more real. For my Master's thesis, I built a data pipeline to construct and process a cross-national research database from multiple sources. It had a clear purpose: take heterogeneous input data, transform it consistently, validate important assumptions, and produce the dataset I needed for the analysis. And it worked. But this is no longer enough. I am not rebuilding it because the original system failed. I am rebuilding it because the question changed: My Master's thesis needed a pipeline. My PhD will need research infrastructure. And I am slowly discovering that these are not the same thing. A pipeline can be finished There is something comfortable about building software for a well-defined research project. You know the research question. You know most of the variables you need. You know which datasets are involved. You can define the transformations, produce the outputs, validate them, run the analysis, and eventually say: Done. Of course, research is never really that clean. Data sources change. Weird edge cases appear. A country disappears from one dataset. Another source changes a variable name. An indicator turns out to mean something slightly different than you thought. But there is still a boundary around the problem. A PhD changes that boundary. Now I have to think about a system that may need to survive several years of research, new questions I have not formulated yet, datasets I have not discovered yet, and methodological decisions I will probably reconsider more than once. Suddenly, "Does it work?" becomes a surprisingly weak design criterion. The more useful

By Michael Nocito , data analyst · Published August 8, 2026 By the end of this page you can turn transactions into a monthly series, add period-on-period change and a cumulative total, get a share-of-total column, smooth a noisy line, and run all of it separately for every group. It is about twenty-five minutes, and every number below came out of running the code. Here is what to do today, on the series you already have. Count its rows against the number of periods in your date range. If your data covers January to May and the series has four rows, a period produced nothing, it never became a row, and every change figure after the gap is comparing the wrong pair. The short version: pct_change() divides each value by the one in the row above; cumsum() adds everything up to and including the current row. Both trust the rows you gave them to be the periods you meant. What happens when the previous period is zero is the idea, so it gets the picture. The original carries a diagram here. In words: Three bar positions stand on a baseline, labelled Mar, Apr and May. The March position holds a tall bar and the May position holds a slightly shorter tall bar. The April position holds no bar at all; there is only a short flat mark sitting on the baseline where a bar would start, drawn in amber to show a value of zero. An arc runs from the top of the March bar down to the April mark, and the figure minus one hundred percent is printed on it, which is a perfectly ordinary answer. A second arc runs from the April mark up to the top of the May bar, and the symbol printed on that one is not a percentage at all but the sideways figure eight that means infinity. The picture shows that a fall to nothing has an answer and a rise from nothing does not. Every number on this page is real. The sixteen-row orders table used across this whole set of guides, run in pandas 3.0.2. It runs from 5 January to 25 May 2026 and contains no April orders at all, which is not staged for this page; it is

By Michael Nocito , data analyst · Published August 8, 2026 By the end of this page you can attach columns from one DataFrame to another on a shared key, choose the right how for the question, see at a glance which rows failed to match, and catch the failure that quietly inflates every total in the frame. It is about twenty-five minutes, and every output below was produced by running the code. Here is what to do today, on every merge you write. Print the row count immediately before and immediately after it. A left merge must not change the row count, and if it did, the right-hand table has the key more than once and your totals have just gone up. The short version: merge pairs rows from two frames wherever their keys match, and the number of rows that come out depends on how many times each key appears on each side. One key twice on the right is the idea, so it gets the picture. The original carries a diagram here. In words: On the left a single row is drawn as a wide box, holding the key Desk and the value 880. To its right stands a small lookup table with two rows, and both of those rows carry the same key, Desk. Two lines run from the single left-hand row, one to each of the two matching lookup rows, so the one row is paired twice. On the far right the result is drawn as two separate output rows, and both of them contain Desk and 880; the value 880 is ringed in amber in each of them to show that it is the same original figure appearing twice. One row went in and two came out, without anything being added to the left-hand table. Every output on this page is real. Sixteen orders totalling 9,890 and a three-row product table, the same tables used across this whole set of guides, merged in pandas 3.0.2 with the results copied back. If you know SQL joins , this is the same operation with different words, and the two failure modes are identical. 1. merge in one line Two frames, one shared column, one call. orders.merge(products, on="product", how="left") order_id prod

Many engineers believe Common Table Expressions (CTEs) are always faster than subqueries. In modern SQL Server (and PostgreSQL), that is a myth . Here is what actually happens under the hood: 1. Inlining & The Query Optimizer By default, the SQL optimizer treats standard CTEs and derived tables (subqueries) almost identically: The engine expands both into the same relational tree. They generate the exact same execution plan and I/O cost . -- Pattern A: Derived Table (Subquery) SELECT DeptID , EmpName , Salary FROM ( SELECT DeptID , EmpName , Salary , DENSE_RANK () OVER ( PARTITION BY DeptID ORDER BY Salary DESC ) AS rnk FROM Employees ) RankedData WHERE rnk <= 2 ; -- Pattern B: Common Table Expression (CTE) WITH RankedData AS ( SELECT DeptID , EmpName , Salary , DENSE_RANK () OVER ( PARTITION BY DeptID ORDER BY Salary DESC ) AS rnk FROM Employees ) SELECT DeptID , EmpName , Salary FROM RankedData WHERE rnk <= 2 ; 2. When CTEs Truly Win: Readability & Pipeline Stacking: You can chain 5 CTEs sequentially without deeply nested pyramid brackets. In-Place Deduplication: In SQL Server, you can run DELETE directly on a CTE, and it deletes duplicate rows straight from the real underlying table! WITH DuplicateCleaner AS ( SELECT CustomerID , Email , ROW_NUMBER () OVER ( PARTITION BY Email ORDER BY RegistrationDate ASC ) AS rn FROM Customers WHERE Email IS NOT NULL ) DELETE FROM DuplicateCleaner WHERE rn > 1 ; -- ✅ Clean in-place deletion! 3. The Big Trap (Spooling Overhead): If you reference the same CTE multiple times in a query (e.g. CTE_A JOIN CTE_A ), SQL Server may execute the underlying CTE query multiple times or create a Lazy Spool in tempdb . -> Fix: For heavy multi-million row reuse, use a Temporary Table ( #TempTable ) with an explicit Clustered Index instead! 💡 How do you choose between CTEs, Temp Tables, and Subqueries in your pipelines? 💼 Connect on LinkedIn: linkedin.com/in/arpitmbangre

What Is Precision Tracking Radar? Precision tracking radar is an active radar sensing system designed to repeatedly measure a selected target and maintain an updated estimate of its state over time. For developers, the important distinction is that precision tracking is not simply repeated target detection. Detection answers: Is there evidence of a target in the current radar measurements? Tracking answers: Does this new measurement belong to an existing target, and how should that target state be updated? A practical precision tracking pipeline can be represented as: RF sensing → target measurement → detection → association → state update → continuous track → mission output That makes precision tracking radar a real-time data-processing system as much as an RF sensing system. A Practical Definition Precision tracking radar is a radar capability that combines repeated target measurements across time to maintain a continuous estimate of target position, motion or other relevant state information. The key word is continuous. A detector can operate independently on each radar update. A tracker has memory. It maintains information from previous measurements and decides how new observations relate to that history. From a software architecture perspective, tracking introduces persistent state into the sensing pipeline. Detection and Tracking Should Be Separate Services A useful radar architecture keeps target detection and target tracking logically separate. The detector processes current radar measurements. The tracker consumes target-related measurements over time. Conceptually: Radar measurement ↓ Detection ↓ Measurement object ↓ Association ↓ Track update ↓ Track state This separation helps developers understand where errors originate. If the detector produces unstable measurements, the tracker cannot fully repair them. If detections are stable but tracks switch between targets, the problem may exist in association. If sensor-relative detections are correct but missio

AI agents are becoming first-class citizens in developer workflows. They can read code, run tests, and deploy apps. But one thing they struggle with is understanding database schemas. Database design tools haven't changed in 20 years. You either use a heavyweight desktop app (Navicat, PDManer) or a pretty but closed web app (dbdiagram). Neither supports versioning, real-time collaboration, or AI agent integration. I built ERD Online to solve this. It's an open-source database design tool that combines Git-like versioning with Figma-like collaboration, plus MCP integration for AI agents. In this article, I'll show you how to let Cursor, Claude, or Cline read and write your database schema through MCP, while you keep full control. Database schema changes are hard to track: Who changed what? When did they change it? Why did they change it? How do I rollback? And now with AI agents, there's a new problem: how do you let an AI agent suggest schema changes without giving it a black box that generates random ER diagrams? The wrong approach: ask AI to "generate an ER diagram for an e-commerce app." You get a diagram, but it has no connection to your actual project, no versioning, and no approval flow. The right approach: let the AI agent read your existing schema, suggest changes, and submit them as a version that you review and approve. That's what ERD Online + MCP does. MCP (Model Context Protocol) is a protocol for AI agents to interact with external tools. Think of it as a USB-C port for AI applications. It standardizes how agents discover and call tools. MCP has three main primitives: Tools : Functions the AI can call (like list_projects or create_version ) Resources : Data the AI can read (like project.json ) Prompts : Pre-defined templates for common tasks ERD Online exposes MCP tools that let AI agents: list_projects : List all your ERD projects get_project : Get a project's projectJSON create_version : Suggest a new version of your schema The key boundary: AI agent

California residents have a legal right to access the data that companies collect about them. Actually exercising that right is a burdensome nightmare.

If you run ClickHouse in production, you're probably on 26.3 LTS. And now 26.8 LTS has been announced, which means the LTS-to-LTS upgrade conversation starts again. Here's the thing most release posts skip: this is not a one-release hop. Going from 26.3 LTS to 26.8 LTS means crossing 26.4, 26.5, 26.6 and 26.7 as well. Every breaking change in those four releases applies to you, and some of the ones most likely to ruin your day aren't in 26.8 at all. So instead of writing another "here are the 26.8 features" post, I wanted to write the thing I'd actually want before scheduling this upgrade: what breaks, what silently changes, what order to do things in, and what you get for the trouble. A note on release timing As of writing (27 August 2026), 26.8 has been announced but is not fully released yet. The release branch is cut and versioned (v26.8.1.1-lts), but the tag and Docker images have not been published yet, and the upstream changelog still marks the 26.8 section as in progress. By the time you read this, the tag has probably landed. Check for yourself: curl -s https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/utils/list-versions/version_date.tsv \ | awk -F '\t' '$1 ~ /^v26\.8\./ {print "26.8 is released - newest: " $1 " (" $2 ")"; f=1; exit} END {if (!f) print "26.8 not released yet"}' version_date.tsv is the list ClickHouse maintains of every released version and its date, so this is the most direct answer available - no auth, no rate limit, nothing to download. As of writing it prints 26.8 not released yet . Worth knowing: the Docker image will lag whatever that command tells you. The Docker Official Images repo trails the GitHub tags by a few patch versions - clickhouse:lts currently resolves to 26.3.20.7 even though 26.3.24.4 has already shipped. So don't treat a missing image as evidence the release hasn't happened. Either way, the timing works in your favour. Historically ClickHouse LTS releases pick up several patch releases quickly - 26.7 had

Most "is this plugin safe?" advice is vibes. I wanted numbers, so I built a dataset. Here's what it found, and exactly how, so you can check my work or build your own. The finding first Of 8,010 WordPress plugins with a publicly documented vulnerability since 2023 (15,534 vulnerability records in total): 3,780 have been removed from the wordpress.org plugin directory. Removal stops updates but doesn't uninstall — affected sites keep running the code. 277 carried a critical (CVSS ≥ 9.0) flaw on record before removal. 2,115 are still installable today with a known vuln and no update in 12+ months — roughly 6.7M active installs combined. The part that surprised me most: "removed from the directory" is nearly invisible to a site owner. No dashboard warning, no email. The plugin just quietly stops getting fixes while sitting on the site. How I built it (no paid APIs) The whole thing runs on two public sources and no API keys. 1. Vulnerability data — the GitHub Advisory Database. It mirrors CVE records including the Patchstack and Wordfence CNA assignments that cover almost all WordPress plugin CVEs. It's a git repo, so a shallow, sparse clone of the advisories/unreviewed/{year} folders gets you the raw JSON: git clone --depth 1 --filter = blob:none --sparse \ https://github.com/github/advisory-database.git Each advisory carries the CVE ID, a CVSS vector string, CWE IDs, and reference URLs. The plugin slug isn't a first-class field — you recover it from the Patchstack/Wordfence reference URLs with a couple of regexes. That alone attributes the large majority of WordPress advisories to a specific plugin. 2. Maintenance signals — the wordpress.org plugin API. For each slug: https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&request[slug]=SLUG That gives install count, last-updated date, tested-up-to version, and support-thread resolution ratio. A 404 (or an {error} body) means the plugin isn't in the directory — but that's ambiguous: it could be removed ,

Most football data scrapers on the market only extract high-level final scores (e.g. 2-1 ). But quantitative sports analysts, data scientists, and predictive betting modelers need granular data: Expected Goals (xG) , Official Referee Assignments , Goal Scorers paired with Assist Providers , and Half-Time vs Full-Time (1H/2H) statistical breakdowns . When I set out to build a professional-grade Flashscore scraper on Apify, I ran into two major engineering challenges: The Memory Problem : Keeping Puppeteer running to scrape hundreds of historical matches consumes over 1.5GB of RAM per run. The Protocol Problem : Flashscore serves its deep statistical feeds using a proprietary pipe-delimited data format ( ~ , ¬ , ÷ ) over CDN endpoints, rather than standard REST APIs. In this tutorial, I'll explain how I engineered the Flashscore Elite Statistics Extractor , how the hybrid Browser + HTTP/2 streaming pipeline drops RAM footprint from 1.5GB to 70MB , how to parse Flashscore's custom feed protocol, and how to pipe the resulting datasets directly into Python and Pandas. 🏛️ The Hybrid Pipeline Architecture To achieve zero proxy reliance for standard runs and ultra-low compute costs, the Actor splits execution into a 2-Phase Hybrid Pipeline : [ League & Season Selection ] │ ▼ ┌───────────────────────────────────────────┐ │ Phase 1: Browser Handshake (Puppeteer) │ │ - Captures x-fsign security tokens │ │ - Extracts countryId & tourId │ └─────────────────────┬─────────────────────┘ │ [ Immediate Browser Shutdown ] (RAM drops from 1.2GB -> 70MB) │ ▼ ┌───────────────────────────────────────────┐ │ Phase 2: Parallel HTTP/2 Feed Workers │ │ - got-scraping with JA3 TLS matching │ │ - Decodes df_st_1_ (Stats) & df_sui_1_ │ └─────────────────────┬─────────────────────┘ │ ▼ ┌───────────────────────────────────────────┐ │ Self-Healing Recovery Pass │ │ - Auto-retries skipped/failed matches │ └─────────────────────┬─────────────────────┘ │ ▼ ┌───────────────────────────────────────────┐

Meta has detailed MTIA 300, its first in-house accelerator optimized for training ranking and recommendation models. By Matt Foster

Startups building B2B products reach for multi-tenancy in PostgreSQL the same way on day one: one shared database, one set of tables, and a tenant_id column marking who owns each row. That is the correct call, and it stays correct for a long time. However, when that column is enforced by application code rather than by the database, a single forgotten predicate stops being a bug and becomes a disclosure event, and a disclosure event is one of the very few engineering failures that lands straight on your balance sheet as stalled enterprise deals, an unplanned legal bill, and a security review you can no longer pass. By understanding what multi-tenancy actually guarantees, which isolation model fits your stage, and how Row-Level Security moves that guarantee out of your codebase, startup CTOs and Fractional CTOs can make the tenant boundary hold without slowing the team down. (If you want to skip the theory, jump straight to the connection pooler trap that switches Row-Level Security off in production, what it costs in query performance, or when it is genuinely time to leave the shared schema.) Because "enforced by application code" means something very specific in practice. It means a promise that everyone will remember to filter on tenant_id , and that promise is the single most expensive line of undocumented policy in your entire codebase, because it holds perfectly for about fourteen months, right up until the afternoon a tired engineer ships a reporting endpoint that joins four tables and forgets the predicate on exactly one of them, and then a customer opens a dashboard and sees somebody else's invoices. That is not a bug. A bug is something you fix on Monday. A cross-tenant data leak is a disclosure event, which means legal gets involved, your enterprise prospects get an email from their own security team, and the deal that was supposed to close your Series A quietly moves to next quarter and then to never. The uncomfortable part is that this is not a story abo

Classic Machine Learning Through the Eyes of an SRE — Part 9: Isolation Forest The algorithm in one line: Isolation Forest scores how anomalous a point is by how few random cuts it takes to separate that point from everything else. No model of normal, no loss function, nothing optimized. ← Previous: Part 8 — Hierarchical Clustering Fails Beautifully · Next: this is the series finale — start at Part 1 . Every anomaly detector I had studied models what NORMAL looks like, then calls the leftovers outliers. K-Means: far from every centroid. DBSCAN: in the noise bucket. Sensible, and intuitive. Isolation Forest does not bother. It never models normal at all. It goes straight at the rare points with a single question: how few random cuts does it take to isolate you? Random cuts, literally. Pick a feature at random, pick a split value at random between that feature's min and max, repeat. A point that separates from the crowd in three cuts is anomalous. A point buried in the middle of a dense mass takes thirty. Grow hundreds of these random trees, average the isolation depth for each point, and you get an anomaly score. There is no loss function here. No optimization, not even the local kind that decision trees do at every split. Every cut is a coin flip, and the power comes entirely from averaging, which is the forest trick from the supervised half of this series now applied to pure randomness. Cheap randomness plus averaging beats careful modeling, as long as the target is something randomness naturally exposes. Rarity is exactly that. Sometimes the winning move is to optimize less. That sentence would have gotten me laughed out of my first ML study session. It is also this finale's thesis. The part I had completely backwards Here is the thing I did not know until I read the original paper properly, and it is the opposite of every instinct a decade of ops gave me. Isolation Forest deliberately trains each tree on a small subsample of your data, and this is not a performan

At TechCrunch Disrupt 2026, the AI Stage is back to dig into the single hottest topic in the community for the past few years, presented by Google for Startups.

Hoomanely has developed a smart bowl to measure and record dogs' feeding data, then tells owners if behaviors change.