标签：#machine

Automatic top-up — the feature that charges a saved card when a customer’s balance falls below their threshold — could never have succeeded for anybody. The invoice was constructed in the wrong currency, and every attempt would have failed in a way that told the customer their card was bad. This is the whole write-up, in the shape we would want any incident written in. Summary An invoice does not take its currency from the line items attached to it. It takes it from the customer’s default currency, or failing that from the Stripe account’s — which is EUR for a Dutch business. Every price in this product is denominated in USD. Finalising the invoice therefore failed with a currency-conflict error, on every automatic top-up, unconditionally. The manual top-up path was never affected, because a Checkout Session takes its currency from the first line item rather than from the customer record. That difference is why the bug could exist in a product whose payment flow demonstrably worked. Impact Dimension Description Customers affected None. The defect was found before the path carried real traffic. This is stated plainly rather than omitted, because a postmortem that lets a near miss read as an outage is as dishonest as one that hides an outage. What would have happened Every automatic top-up fails. The failure surfaces as a payment error, which the failure counter records as a strike, and after three strikes the customer's automatic top-up is switched off entirely. What the customer would have concluded That their card was declined. The message they receive says the saved card could not be charged. They would have gone and fixed a card that was working perfectly. Secondary effect A customer relying on automatic top-up to keep a production integration serving would have run out of credit silently, at whatever hour their traffic happened to cross the threshold. The second and third rows are what make this worth writing up. A defect that fails loudly and correctly is a bug

“AI discovered a new material.” “AI found a drug candidate.” “AI solved protein folding.” Each of those sentences can be true, badly misleading, or flatly wrong depending on one thing the sentence does not tell you: how far the result got from the model before somebody wrote it down. The sentence that hides four different claims Take a single headline: a model proposed a molecule that binds a protein implicated in a disease. That sentence is compatible with at least four very different states of the world. The molecule might exist only as a string in a file. It might have been synthesised. It might have bound the protein in a test tube. Or it might have improved an outcome in a person. Those four are separated by years, by orders of magnitude in cost, and by a probability of success that drops at every step — and press coverage routinely reports the first as though it were the fourth. This is not a complaint about journalism. It is the single most useful thing to internalise about the whole field, because once you have the ladder in your head you can grade a claim in about ten seconds, and you can do it for a subject you know nothing about. The ladder The rungs are the same in every discipline. Only the names of the instruments change. Rung Description 1 · Output The model emitted something: a structure, a score, a candidate, a forecast. Nothing has been checked. Everything downstream is conditional on this being worth checking. 2 · Retrospective The output was compared against data that already existed — held-out structures, historical weather, known compounds. This is where nearly all published numbers live, and it is entirely dependent on the held-out set resembling the future. 3 · Prospective The prediction was made first and the answer arrived afterwards. A forecast verified against what the weather then did. A candidate synthesised after being proposed. This rung is qualitatively stronger than rung 2 and much rarer. 4 · Confirmed An independent method establis

Every team already knows how to put a feature behind a flag. What is different here is that the thing most likely to need changing at three in the morning is not whether the feature is on — it is which model it calls, which prompt it uses, and how much it is allowed to do without asking. Why the usual flag is not enough A conventional feature flag answers one question with a boolean, and it is the right shape because a conventional feature has one failure mode: it is broken. An AI feature has several, and they want different responses. The provider is degraded — you want a different model, not the feature off. A prompt change regressed quality — you want the previous prompt, which is not a code deploy. The feature is fine but a specific customer’s data is producing bad output — you want it off for them and on for everyone else. Spend is running above forecast — you want the cheap model or the degraded path, not an outage. A single boolean answers none of these, so the response to each becomes a deploy, and a deploy is the slowest tool available at the moment you most need speed. There is a second reason, specific to this dependency. The behaviour you are flagging can change without you deploying anything, because the model is somebody else’s and it can be updated underneath you. Flags are usually a mechanism for controlling your own changes; here they are also the mechanism for reacting to changes you did not make, which is why detecting a provider-side behaviour change and having a flag to respond with are two halves of one control. Four things to flag separately Axis Description Feature on/off The ordinary flag. Per-tenant and per-segment, because the common case is a problem confined to one customer's data rather than a global outage. Model selection Which model each call site uses, as configuration. This is what lets you switch providers during an incident, run a canary on a new model, or drop to a cheaper one under budget pressure — without shipping code. Promp

“Something went wrong. Please try again.” is correct for about a third of AI failures and actively harmful for the rest, because for the rest, trying again cannot possibly help and you have just told the user to spend money finding that out. Everything that can go wrong Errors arrive from at least four layers, and the user-facing consequences differ enough that collapsing them into one message destroys the only information you had. Failure Description Transport Connection dropped, DNS, TLS, the stream died mid-token. Retryable, usually transient, and the user did nothing wrong. This is the only class where 'try again' is straightforwardly true. Rate limited (429) Yours or the provider's capacity, not the request. Retryable but only after a wait, and the wait is often stated in a header. Telling the user to retry immediately guarantees a second 429. Provider 5xx / overloaded Retryable with backoff, and the single best case for automatic failover to another provider rather than for any message at all. Timeout Ambiguous by construction: the request may have completed on the provider's side and been billed. Retrying may duplicate a side effect, which is why idempotency matters more here than anywhere. Context length exceeded Deterministic. Retrying the identical request fails identically. The only fix is fewer tokens, and the interface knows that — so the message should offer the fix, not the retry. Content filter The provider blocked the input or the output. Not retryable unchanged. Distinct from a model refusal, and users experience the two very differently. Truncated output The generation hit max_tokens. Not an error at the transport layer at all — status 200, a finish reason of 'length', and an answer that stops mid-sentence. Silently the most common broken experience. Malformed structured output Valid HTTP, invalid JSON or a schema violation. Retryable and often succeeds on a second sample, because it is a sampling accident rather than a capability failure. Empty o

Estimates of the energy in one model query differ by orders of magnitude across credible sources. Most of that spread is not disagreement about physics. It is disagreement about where the system boundary is drawn, and a figure quoted without its boundary is not a figure. Why the published figures disagree Before comparing two numbers, establish which of these each one includes. Any of them can change the answer by more than the model choice does: Which model, and how much output. A short answer from a small model and a long answer from a large reasoning model differ by several orders of magnitude on their own. A single “per query” figure averages over a distribution nobody specifies. Batch size and utilisation. The dominant engineering term. Serving many requests concurrently amortises the weight read across all of them; the same hardware at low occupancy spends nearly the same power for a fraction of the tokens. Facility overhead. Cooling, power conversion and distribution, captured as power usage effectiveness. It multiplies everything, and whether a figure includes it is frequently unstated. Training amortisation. Some analyses divide training energy across expected lifetime queries. Defensible, and it produces a different quantity from marginal serving energy. They are not comparable. Embodied energy. Manufacturing the accelerators, the building and the power infrastructure. Usually excluded, occasionally included, rarely flagged. Idle and provisioned capacity. Capacity is held for peak. Charging queries only for the seconds they compute understates the system; charging them for provisioned capacity overstates the marginal query. Both are used. Building the estimate yourself The marginal serving calculation is not complicated, and doing it once makes every published figure legible. Serving side, per accelerator: E_per_token = (P_device · n_devices · PUE) / R_tokens_per_second P_device average power draw under load, from the spec sheet (below the rated maximum in

Economists modelling AI reach conclusions ranging from a modest productivity bump to a change in the growth regime. They are not using different data. They are using different values for three or four parameters, and the parameters are where the argument should be. Two families of model Task-based automation models Associated most closely with Daron Acemoglu and Pascual Restrepo, these treat production as a continuum of tasks, each performed by labour or by capital. Automation moves tasks from labour to capital, which raises productivity and displaces workers; new task creation moves the boundary back. Growth and distributional effects both fall out of the movement of that boundary. The framework’s virtue is that it makes the aggregate effect an explicit function of quantities you can in principle measure: what share of tasks is exposed, how much cost is saved on each, and how fast new tasks appear. Acemoglu’s own applications of it to AI produce deliberately conservative aggregate numbers, and the reasoning is transparent — the effect is bounded by the exposed share times the saving on that share, so a large aggregate effect requires both terms to be large. Idea-production models The semi-endogenous growth tradition, associated with Charles Jones, models growth as driven by ideas, with ideas produced by researchers. Its central empirical observation is that ideas are getting harder to find: research effort has risen dramatically while growth has not, so productivity per researcher is falling. Aghion, Jones and Jones applied this framework to AI directly, and the key move is that AI enters not as a better tool but as a substitute for researchers themselves. That changes the mathematics qualitatively rather than quantitatively. If the population of effective researchers can be expanded by producing more compute rather than by waiting for demographic growth, the constraint that keeps growth steady in these models is loosened, and under some parameter values the models

This page is about method, not about any particular medicine, and nothing here is medical advice. It is written to answer one question: when a company says a drug was discovered with AI, which part of a decade-long process is that sentence about? The pipeline, and where the years go Roughly, and with enormous variation: pick a target, find molecules that do something to it, optimise those molecules into something drug-like, test in animals and in safety assays, then run the clinical stages — first for safety in a small number of people, then for efficacy in patients, then in a large confirmatory trial — and then apply to a regulator. Start to finish is usually over a decade. Two facts about that pipeline determine everything else on this page. The first is that the calendar and the money are dominated by the clinical stages, not the discovery ones. The second is that failure is the normal outcome, and it is concentrated where the drug first meets human biology: a candidate can be a beautiful molecule, hit its target exactly as designed, and still not help anyone, because the target was the wrong thing to hit. Where models are genuinely used Application Description Virtual screening Score enormous make-on-demand chemical libraries against a target site far faster than physics-based docking can. The output is a shortlist to synthesise and assay, and it replaces a search, not an experiment. Generative chemistry Propose molecules conditioned on a target, a scaffold or a set of property constraints, rather than picking from a catalogue. Whether the molecule can be made at all is a separate model. Property prediction Solubility, permeability, metabolic stability, cardiac ion channel liability. These filter a list early and cheaply. They are trained on assay data and inherit its coverage: they are most reliable on chemistry that resembles what has been tested. Retrosynthesis Plan a route from purchasable starting materials. This is the application closest to a solved probl

I’m curious whether there is now a theoretical or empirical “sweet spot” for LLM quantization, preferably research done using open-source formats like GGUF Suppose you have a fixed memory/compute budget and can choose the model size freely. For example, instead of a smaller model at 8-bit or 4-bit, you could fit a progressively larger model at 3-bit, 2-bit, 1.5-bit, etc. A few years ago, I remember 4-bit often being described as roughly the practical sweet spot because it preserved most model quality while giving a large memory reduction. But with newer methods, I’ve seen surprisingly strong 3-bit, 2-bit, and even ~1.5-bit results. So if the goal is maximum model capability for a fixed memory budget , rather than preserving one particular pretrained model as faithfully as possible, what does current research suggest is the optimal bits-per-weight? Is there evidence that, for example, a 2-bit 70B model generally beats a 4-bit 35B model, or does quantization degradation eventually outweigh the gains from additional parameters? I’m especially interested in recent theoretical/scaling-law work or large empirical studies from 2025–2026. If no one is studying this, then could any of you do this work? I feel like it could be immensely useful for the community. submitted by /u/takuonline [link] [留言]

To all those in the US: Are you planning to go Sydney or Atlanta this year for NeurIPS? submitted by /u/rsesrsfh [link] [留言]

Table of Contents Overview AI Didn't Remove the Skill, It Relocated the Skill Vibe Coding...

It's an MLP architecture with around 500K total parameters. Top1 Training accuracy: 5.11% Validation accuracy 4.59% Detailed Validation accuracy numbers: Top-1 Acc: 4.59% Top-3 Acc: 9.44% Top-5 Acc: 12.68% Top-10 Acc: 18.53% The model was trained on a downscaled version of the Imagenet-1k dataset (32x32) for 5 epochs. I used pytorch for the training and pyarrow for the dataset, all within termux. Before anyone comes at me for using an MLP instead of a CNN or similar it's mainly because on my phone an MLP was just more stable, and trained 10-30x faster/step (could be my fault but I'm not too sure). This model specifically took around 30 minutes to train (6 minute/epoch) The training was entirely on the CPU which is a Dimensity 9300+ and I used 4 of the Arm Cortex-X4 cores. I might make an improved version later on as this one isn't very accurate. submitted by /u/Tall_Abrocoma_3533 [link] [留言]

Every anomaly detector answers one question: compared to what? That comparison, the baseline, is where I lost the most time on this project, and every failure had the same signature. Nothing errored. No test went red. The numbers stayed plausible. The detector just quietly stopped detecting. Four of them, in the order I found them. 1. The peer group contained the client it was judging Cold-start clients have no history, so they're compared against a pool of other clients' recent benign windows. Reasonable. The pool was keyed by feature: private readonly peer = new Map < FeatureKey , number [] > (); Every benign window every client produced went into the pool that client was later compared against. Including itself. So a client could define its own normality . Feed in enough windows and any behaviour becomes unremarkable — which is precisely the cold-start attacker the layer exists to catch. What made me look was not reasoning, it was an experiment that wouldn't sit still. I was trying to build a demo client that reliably landed in the middle of the response ladder, and holding the traffic shape fixed while changing only the request interval flipped the outcome between allow and step_up : gap=500ms origins=5 → allow (peak 0) gap=700ms origins=5 → step_up (peak 83) gap=800ms origins=5 → allow (peak 0) A knife edge like that is never a tuning problem. The outcome depended on a race between a client's own samples reaching the pool and the pool being consulted. Fix: key the pool per client, and exclude the client under evaluation. for ( const [ clientId , values ] of byClient ) { if ( clientId === excludeClientId ) continue ; // this is what "peer" means … } Afterwards the behaviour became monotone in the actual evidence, and identical at every request interval: origins 1 3 4 5 6 peak score 17 35 59 83 100 tier allow log throttle step_up deny Lesson: if a parameter that shouldn't matter changes the outcome, stop tuning and go find the defect. Knife edges are symptoms. 2.

Features get you a vector per window. Turning that into a decision is where the design choices are. This system scores every window three independent ways and takes the strongest single case. Each layer covers a failure mode of the others. Layer 1: guardrails Deterministic thresholds, no baseline of any kind: // Honeytoken hit — highest-confidence signal. Immediate revoke. if ( fv . honeytoken_hits > 0 ) add ( 100 , ' honeytoken_hits ' , ' … ' ); // High miss ratio — guessing IDs that mostly do not exist. if ( fv . miss_ratio >= 0.4 && fv . req_count >= 10 ) add ( 88 , ' miss_ratio ' , ' … ' ); // Sequential walk — near-adjacent IDs in order. if ( fv . id_sequentiality >= 0.8 && fv . distinct_resource_ids >= 10 ) add ( 90 , ' id_sequentiality ' , ' … ' ); // Working set that expands and never stops — the mimicry signature. if ( fv . window_size === ' 1m ' && fv . novelty_run_length >= 20 ) add ( 86 , ' novelty_run_length ' , ' … ' ); Being baseline-free is the point: they fire on a client's first window. A statistical layer needs history to say anything, so a brand-new compromised integration, one that never had a quiet period to learn from, is invisible to it. Guardrails cover exactly that gap. The deliberate omission is cardinality. There is no "distinct IDs > N" guardrail in the scorer, for the reasons in part 3 : it false-positives on legitimate bulk reads and no threshold fixes that. (The gateway's fast path does have a cardinality rule, at 150 distinct/minute — well above any realistic backfill, and it exists to stop a flood before the first window closes.) Layer 2: robust statistics Per client, per feature, per window size: keep a bounded history and score new values with a median/MAD robust z-score . Median and MAD rather than mean and standard deviation, because mean and σ are themselves distorted by the outliers you're hunting. One 5,000-request window drags a mean enough to make the next one look normal. Three things make this work in practice, and each w

I played a bit with the SIREN network from the other post and found that it could be improved by a using a different sampler for batch generation. By feeding pixels across the entire video and not only a limited set of frames, we can a much more faithful reproduction of the video. The model is exactly the same as used by OP: 4 x 512 wide sine layers, 792257 parameters. Its a reimplementation (using GPT5.6). I also created a version with full framerate, instead of subsampled frames, but since the network has to memorize more temporal information, the image reconstruction suffers compared to the low rate version. The model does not actually learn motion, intermediate frames are nonsensical. I suppose adding a layer that can model flow between frames could enhance the compression a lot. You can find the code here in this gist . I tried some addition experiments with a separate autoencoder to compress the frames separately. This resulted in a smaller model, but also degraded quality. submitted by /u/cpldcpu [link] [留言]

CIKM 2026 decisions will be announced today. The resource track outcomes have started going out. How did you go with CIKM 2026? submitted by /u/Happy-Hustler [link] [留言]

Classic Machine Learning Through the Eyes of an SRE — Part 3 The random forest is the first ML algorithm that made me feel at home. Not because of the math — because it's an SRE idea wearing a stats costume. Many independent workers. No single point of failure. Majority vote. If one worker goes weird, the fleet absorbs it. We've been building systems this way for decades; the forest just applies it to prediction. The problem it exists to fix Last article: a single decision tree is readable but unstable — small data change, whole tree flips, explanation rewrites itself. That instability is variance, and it's exactly what scared me about trusting one tree in production. The forest's move: grow hundreds of trees, each on a random resample of the data, and — this is the part that matters — force each split to choose from only a random subset of features. That second randomization is the whole difference between a random forest and plain bagging. Bagging alone gives you many trees on resampled data, but if one feature is strongly predictive, every tree grabs it first and they all end up looking alike. Starving each split of features is what makes the trees genuinely different from each other. The randomness isn't sloppiness. It's manufactured disagreement. The instability doesn't get fixed. It gets CANCELLED. Each tree is still jumpy, but they're jumpy in different directions, and the average is calm. What surprised me No new loss function. Each tree still minimizes impurity exactly like a lone tree. The forest adds zero new objectives. The entire gain is a bias-variance bargain: variance drops hard, bias barely moves. You give up readability and get back trustworthiness. Embarrassingly parallel. Trees are independent, so training scales horizontally — throw cores at it. Boosting, its sequential cousin, is the opposite: each model depends on the last. Map-reduce versus a pipeline. The smoothness illusion. A forest's decision boundary looks smooth, almost like regression'

The results are out today! Let’s share them, guys. From my batch - 3/6 full papers - 1/3 short papers are accepted Cheers! submitted by /u/snu95 [link] [留言]

We’ve all been there: you're crushing your workouts, feeling like a beast, and then suddenly— bam . You can’t get out of bed, your resting heart rate is through the roof, and your motivation has evaporated. Welcome to Overtraining Syndrome (OTS) . In the world of sports science, Heart Rate Variability (HRV) is the gold standard for tracking recovery. By analyzing the tiny fluctuations between heartbeats (R-R intervals), we can peek into our Autonomic Nervous System (ANS). Today, we’re going to build a Python-based pipeline to fetch data from the Oura Cloud API , calculate key HRV metrics like SDNN and RMSSD , and use an Isolation Forest model to detect when you're pushing a bit too hard. Whether you're a biohacker or a developer interested in wearable data analysis , this guide will show you how to turn raw health data into actionable recovery insights. The Architecture: From Pulse to Prediction 🏗️ Before we dive into the code, let's visualize how the data flows from your finger to our anomaly detection model. graph TD A[Oura Ring] -->|Sync| B(Oura Cloud API) B -->|Raw R-R Intervals| C{Data Preprocessing} C -->|Filtering Artifacts| D[Feature Extraction] D -->|SDNN & RMSSD| E[Isolation Forest Model] E -->|Normal| F[Keep Training! 🚀] E -->|Anomaly| G[Rest Day Required! 🛑] Prerequisites 🛠️ To follow along, you’ll need a few tools in your tech_stack : Python 3.9+ Scikit-learn : For our machine learning magic. SciPy/NumPy : For the heavy math lifting. Oura Cloud API Access : To get that sweet, sweet biometric data. pip install scikit-learn scipy pandas requests Step 1: Fetching R-R Intervals from Oura 💍 The Oura Ring records "R-R intervals" (the time between successive heartbeats in milliseconds) during sleep. This is much more granular than a simple "Heart Rate" average. import requests import pandas as pd def fetch_oura_hrv_data ( api_token , start_date , end_date ): url = f ' https://api.ouraring.com/v2/usercollection/heart_rate ' headers = { ' Authorization ' : f ' B

Automatic speech recognition is very good until somebody mentions the name of a local school, a village, a politician, a festival, or a cricket ground. Then things get strange. In an earlier test with audio from Barbados, GPT Transcribe and GPT Audio 1.5 heard the event name “Rise Together” as “Rice Together”, while Qwen3.5-Omni Plus and Flash got it right. Those are different models from the Qwen3-Omni checkpoint used here, but the result motivated this experiment. Acoustically, the mistake is understandable. Culturally, it is wrong. A person who knows the local context has another signal available: they know that Rise Together is the plausible name. That led me to a question: can we give an audio-native model a stronger model of Barbados, using text that already contains the names, institutions, places, events and relationships it is likely to hear? So I took an archive of Barbados newspapers, turned it into 51.6 million tokens, and used it for domain-adaptive pretraining of the Thinker inside Qwen3-Omni. The result is promising, but not conclusive. The adapted model produced higher scores on our preliminary Barbados knowledge probe, particularly on people and institutions. It also got slightly worse on a small set of general-knowledge controls. And, most importantly, we have not yet shown that it transcribes audio more accurately. This is a very preliminary result. It came from our first training run, which we stopped at step 500 of a planned 801 steps. We were also still extracting the newspaper archive, so the 51.6 million training tokens represent the material available for that run rather than the full corpus we ultimately intend to use. This post is about what we have actually demonstrated, what broke along the way, and why I think the experiment is still worth pursuing. The Problem Is Not Just Acoustic A transcription model is doing more than converting sound into letters. When audio is clean and a word is common, the acoustic evidence can be enough. But re

Every failure I am about to describe produced a clean run. No exception, no stack trace, no red build. Each one produced a plausible number that I believed for longer than I should have. That is the category of bug I have come to fear most. A crash tells you it crashed. A silently broken dataset tells you nothing at all, and your metrics will politely agree with it. Here are three from the last year, all from my own work, all found late. 1. The dataset that was 92% one category I had a training set of 688 records for a multi-category vision-language task. Thirteen categories. Reasonable size for a fine-tune, already used in a completed training run whose results I had written up. While preparing a stratified split, I joined the records back against the source annotations and actually counted the categories. 630 of 688 were a single category: scene captions. Zero examples of traffic signals. Zero of planning. Zero of uncertainty. Several categories the evaluation explicitly measured had no representation in training at all. The previous fine-tune had shown gains on some of those very categories. I had interpreted this as the model learning the task. The real explanation was duller and more useful: the model had learned the answer format from caption supervision, and format alignment alone was enough to move a multiple-choice score. Nothing category-specific had been learned, because nothing category-specific had been shown. The root cause was upstream and boring. The conversion script I inherited only rewrote file paths and dropped records with missing frames. It faithfully preserved a caption-only selection made further up the chain. It had no opinion about balance because nobody had asked it to have one. What I changed: the composition of a training set is now an artifact I generate and inspect before any run, not a property I assume. A category histogram takes seconds. I had not looked, for months. 2. The 18-hour run that converged perfectly to nothing Large model