标签：#science

Every team I've watched ship a broken measurement system broke it the same way. Not with bad math — with an org chart problem that happened to live in code. The entity making the claim ended up being the entity that decided whether the claim was right. Once you have the shape in your head you start seeing it everywhere. Three roles, not two Most engineers think about measurement as two roles: the thing that acts, and the thing that grades it. That's one role short. There are three: Player — makes the claim. Your model, your service, your PR. Scorer — applies the rubric. Your eval harness, your test suite, your metrics dashboard. Settler — determines what actually happened. Production outcomes. Reality. The scorer is a proxy. The settler is the thing the proxy is trying to approximate. The rule: be the scorer, never the settler. When the player captures the settler, the loop closes on itself and the system can no longer be wrong — which sounds like success and is actually the failure. What it looks like in code Tuning on the test set. You check test accuracy, adjust hyperparameters, check again. Twenty iterations later the test set is training data with extra steps. The player is now selecting its own settler. That's what overfitting is , structurally — not a math failure, a role-collapse failure. LLM-as-judge from the same family. Your generator is GPT-flavored and your judge is GPT-flavored. They share pretraining data, failure modes, and blind spots. The judge doesn't rate quality — it rates similarity to what it would have produced. Correlated error is invisible to averaging; running it 1,000 times makes you more confident of the same wrong answer. Benchmark contamination. The model scores 94% on the benchmark that's in its training data. Nobody lied. The settler just quietly moved inside the player. Self-reported health. A service that returns its own health check is a claimant ruling on its own claim. If the process is wedged, the check is wedged too, and your

"Preliminary investigation shows that two of Link's three reaction wheels currently are not operable."

Every binomial confidence interval you have ever computed on an eval pass rate, Wald, Wilson, Clopper-Pearson, all of them, rests on one assumption: each example is an independent draw. Most eval sets violate it. You have 40 questions generated from the same 8 documents, or 200 turns from the same 30 conversations, or 150 examples that are really 50 cases with 3 paraphrases each. Those are not 200 independent observations. And when you feed a correlated set into a formula that assumes independence, the interval comes out too narrow, which means you declare differences significant that aren't. I want to walk through why, put a number on how much it matters, and show the fix, because this one is invisible: the code runs, the interval prints, and it is quietly wrong. Why clustering shrinks your real sample size Independent examples each carry their own information. Correlated examples carry overlapping information. If five questions come from the same document, and the model either understands that document or doesn't, those five outcomes move together. You did not learn five independent things about the model. You learned something closer to one and a half. The survey-statistics name for this is the design effect (Kish, "Survey Sampling," 1965). For clustered data it is approximately: Deff = 1 + (m̄ - 1) · ICC where m̄ is the average cluster size and ICC is the intra-cluster correlation, the fraction of total variance that lives between clusters rather than within them. Your effective sample size is: n_eff = n / Deff That is the number of independent examples your clustered set is actually worth. The number Take a realistic eval set: n = 200 examples, drawn from 40 source documents, so average cluster size m̄ = 5. Suppose the ICC is 0.3, which is unremarkable for "questions from the same document" (I have measured higher). Deff = 1 + (5 - 1) · 0.3 = 2.2 n_eff = 200 / 2.2 ≈ 91 Your 200-example eval is worth about 91 independent examples. The correct confidence interval

“We're in the realm where we might be essentially killing off everything within that first hour or two."

Scientists have long assumed that there’s an upper limit to the intensity of the solar winds that reach Earth. New research suggests there’s not—and the implications are alarming.

Extreme fire conditions on the ground have created unprecedented conditions in the atmosphere.

Climate change is upping the odds of major wildfires, creating a need for new tools to combat them.

This series follows a fictional conversation between an experienced engineer and his nephew. Every episode explores one stage of how software moves from an idea to production. 👦 Nephew: Uncle, requirements are clear. I checked the codebase — there's already a FavoritesService I can extend for Wishlist. Now can I open VS Code? 👨‍🦳 Uncle: Almost. Tell me — what do you think HLD even is ? You've heard the term in every interview. What do you think it actually means? 👦 Nephew: Some kind of... diagram? Boxes connected with lines, before you start coding? 👨‍🦳 Uncle: That's what it looks like. That's not what it's for . Let me ask differently. Why do you think experienced engineers insist on drawing this before touching code, when they could just start building? 👦 Nephew: ...to plan the work? 👨‍🦳 Uncle: Closer, but still not it. Here's the real answer: HLD exists to decide, in advance, where the walls go — so that six months from now, when someone adds a new feature, they know exactly which room to build it in, without knocking down a wall that was holding up the ceiling. 👦 Nephew: That's a strange way to describe a diagram. 👨‍🦳 Uncle: Then let me show you, instead of describing it. That's the only way this actually lands. What Talks to What 👨‍🦳 Uncle: Suppose we're building this at Flipkart. Not a college project — a company with hundreds of live services, where breaking one thing can affect ten others you've never even heard of. Here's the simplest picture for Wishlist. Frontend ↓ Wishlist API ↓ Wishlist Service ↓ Database 👦 Nephew: That's it? Four boxes? 👨‍🦳 Uncle: That's it. HLD answers exactly one question, and nothing more — what talks to what. Not how the button looks. Not what fields the database table has. Just: which component calls which, and in what direction. 👦 Nephew: Then why does everyone treat it like it's such a big deal? This took ten seconds to draw. 👨‍🦳 Uncle: Because the value isn't in the ten seconds you spend drawing it today. The value is in what i

Most experiments that come back "no clear winner" were unwinnable on the day they launched. The data could not resolve an effect that size, and no amount of extra runtime was going to change that. You can find this out in about two minutes, before you spend anything, with one formula and a resampling pass over your own data. Here is the check, in three steps. Step 1. Compute the smallest lift your data can see For a two-arm test on a conversion rate, the smallest lift detectable at 95% confidence and 80% power is a one-liner: from math import sqrt Z_ALPHA = 1.96 # two-sided 95% Z_BETA = 0.84 # 80% power def mde ( baseline_cvr : float , n_per_arm : int ) -> tuple [ float , float ]: """ Minimum detectable effect: absolute (pp) and relative (%). """ se = sqrt ( 2 * baseline_cvr * ( 1 - baseline_cvr ) / n_per_arm ) abs_lift = ( Z_ALPHA + Z_BETA ) * se return abs_lift * 100 , abs_lift / baseline_cvr * 100 At a 3% conversion rate: clicks per arm smallest lift you can detect 5,000 +32% relative 20,000 +16% relative 100,000 +7% relative Read the middle row twice. Twenty thousand clicks per arm is a serious amount of traffic for a mid-market account, and a real 15% improvement still lands inside the confidence interval. The report will say "inconclusive," and the team will read that as a verdict on the idea. It is a verdict on the instrument. Invert the same formula and the planning question gets easier: at 3% baseline, detecting a 10% lift needs about 51,000 clicks per arm, and detecting a 5% lift needs about 203,000. If your account produces 8,000 clicks a month, you now know the honest answer to "how long should we run this." Step 2. Stop assuming your conversions are independent The formula above treats every click as an independent coin flip with the same probability. Account data does not behave that way, and the gap is not small. In a corpus of 31 advertiser accounts I maintain for diagnostic work (9.46 million search term rows, roughly $133M of spend, September 2024

Amazon filed an FCC application on Saturday to launch a new Leo satellite constellation that will provide direct-to-device satellite service for "voice, messaging, data, and emergency services." If approved, Amazon will begin deploying the new constellation of 5,105 satellites in 2028. It says it plans to partner with mobile network operators to offer direct-to-device satellite […]

Beluga whales were in danger of getting euthanized after a Canadian theme park went bust. WIRED spoke with some of the scientists behind the groundbreaking rescue mission.

As the US sees its highest number of measles cases in decades and vaccination rates fall, researchers are developing drugs to help those who contract the virus or who are particularly vulnerable.

Black holes eject powerful energy jets that blow away the surrounding gas to great distances. So how can they continue to grow?

Regression and regularization are both important concepts in machine learning and statistics, but they solve different problems. Regression is primarily used to model relationships and make predictions. Regularization is used to improve a model's ability to generalize by controlling its complexity. Regression This is a statistical and machine learning technique used to predict a continuous numerical outcome based on one or more input variables. For example, we might want to predict: A house's price based on its size and location A student's exam score based on study hours A company's sales based on advertising spending Simple Linear Regression In simple linear regression, we model the relationship between an input variable (x) and an output (y): $$ y = \beta_0 + \beta_1x + \epsilon $$ Where: (y) is the predicted outcome (\beta_0) is the intercept (\beta_1) is the coefficient or slope (x) is the input variable (\epsilon) represents the error The model learns values for (\beta_0) and (\beta_1) that make its predictions as close as possible to the actual values. Multiple Linear Regression In multiple linear regression, several predictors are used: $$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \cdots + \beta_px_p + \epsilon $$ The goal is typically to minimize the sum of squared errors (SSE) : $$ \text{SSE} = \sum_{i=1}^{n}(y_i - \hat{y}_i)^2 $$ This approach is known as Ordinary Least Squares (OLS) . Regularization Regularization is a technique used to prevent a machine learning model from becoming too complex. A model can perform extremely well on training data but poorly on new, unseen data. This problem is called overfitting . Regularization addresses overfitting by adding a penalty for large model coefficients to the model's objective function. Instead of minimizing only the prediction error, the model minimizes: $$ \text{Prediction Error} + \text{Complexity Penalty} $$ The penalty discourages the model from relying too heavily on individual features. The Main Types o

More electric vehicles are becoming capable of providing backup power at a time when the US is increasingly in need of more electrons.

"What if satellite data could help protect the livelihoods of millions who depend on Africa's largest lake?" Our team JONAM had the privilege of participating in the Kijani Space Hackathon, where we proudly secured 3rd place while tackling Challenge 2: Sustainable Fisheries & Blue Economy. Rather than building another dashboard, we wanted to solve a real problem affecting millions of people around Lake Victoria: declining fish stocks caused by worsening water quality. Lake Victoria supports millions of people through fishing, transportation, agriculture, and tourism. However, over the years the lake has experienced: Increasing water pollution Poor water quality Frequent algal blooms Reduced fish breeding habitats Declining fish populations For fishing communities, these are not just environmental issues—they directly affect livelihoods, food security, and local economies. Our question became: Can Earth observation data help communities understand where water conditions are becoming unsuitable for fish before the problem becomes critical? Our Solution: JONAM JONAM is an AI-powered web application that combines satellite-derived environmental data with machine learning to monitor water quality and provide insights into conditions that may contribute to declining fish stocks. Instead of relying solely on manual sampling—which is expensive and only covers small areas—our platform continuously analyses satellite observations covering the entire lake. Why Copernicus? To build JONAM, we integrated the KijaniBox API, which provides access to environmental datasets from the Copernicus Programme. Copernicus is the European Union's Earth observation programme. It uses a constellation of Sentinel satellites together with in-situ observations to monitor Earth's atmosphere, land, and oceans. For our project, we focused specifically on live water telemetry variables available through the KijaniBox platform. Water Temperature Satellites measure the thermal radiation emitted from th

SpaceX will likely attempt to catch Starship back at the launch pad on its next flight.

Museums are embracing data-driven curation and a shifting technology landscape.

Electrolyte powders promise better hydration, energy, and recovery. But unless you’re losing serious fluid, water and food probably have you covered.

Scientists identified the first known cancer transmissible among freshwater fish in a lake that spans the US and Canada.