标签：#MachineLearning

Two years ago, "run a language model locally" meant a weekend of compiling, a graveyard of CUDA errors, and a model that answered like it had a concussion. In 2026, you can install one tool, type one command, and have a genuinely useful assistant running on a laptop with no internet connection. Here's an honest map of what works, what doesn't, and where the sharp edges still are. Why bother running locally at all Three reasons keep pulling developers back to local inference: Privacy. The prompt never leaves your machine. For code you can't paste into a cloud box, or personal data, that's non-negotiable. Cost and offline. No per-token bill, no rate limits, and it works on a plane. Latency and control. No network round-trip, and you pin the exact model version forever — no silent upgrades changing your outputs. The catch has always been quality-per-watt. That's the number that moved. The hardware tiers, honestly 8 GB RAM / integrated GPU: You can run 3–4B parameter models at 4-bit quantization. Good for autocomplete, summarizing, simple Q&A. Don't expect deep reasoning. 16 GB RAM: The sweet spot for most developers. 7–9B models run comfortably and are genuinely helpful for coding assistance and drafting. 32 GB+ or a discrete GPU with 16–24 GB VRAM: Now you're running 20–30B models, or bigger models at aggressive quantization, with real reasoning ability. Apple Silicon (unified memory): Punches above its weight. A machine with 32–64 GB of unified memory runs models that would need an expensive discrete GPU on other platforms, because the CPU and GPU share the same memory pool. Quantization: the trick that makes it possible The reason a 7B model fits in 16 GB is quantization — storing weights at 4 bits instead of 16. The common format you'll see is GGUF, and the common recipe is 4-bit (often labeled Q4). The quality loss from full precision to 4-bit is surprisingly small for most tasks, while the memory savings are 4x. Below 4-bit (2–3 bit) the model starts to degrade n

For two years the reflex was simple: reach for the biggest model you can afford and call it a day. In 2026 that reflex quietly became a bug in your cost model. The clearest signal came this summer, when a smaller, cheaper "flash"-tier model started edging out its own flagship sibling on the workload developers care about most — multi-step agentic coding — at a fraction of the price. When the fast tier wins the hard benchmark, "always use the flagship" stops being a safe default and starts being waste. Here's how to fix it without turning your stack into a science project. Why the reflex is expensive Agent workloads are not one big call. A single task fans out into dozens of small ones: planning, tool selection, argument formatting, summarizing a file, deciding whether to continue. Most of those steps are easy . Routing every one of them through a frontier model is like taking a helicopter to the corner store — it works, but you are paying helicopter prices for a walk. The trap is that the cost is invisible per call and enormous in aggregate. You never see the moment you overpaid; you just see the invoice. The three-tier ladder Think in tiers, not models: Cheap/fast tier — classification, extraction, short rewrites, routing decisions, "is this done?" checks. Most steps live here. Mid tier — normal reasoning, code edits, tool use with moderate context. Flagship tier — genuinely hard reasoning, long-context synthesis, the step where a wrong answer poisons everything downstream. The goal is to keep the flagship tier for the 5–15% of steps that actually need it, and let the cheap tier carry the volume. How to decide the tier per request Two mechanisms, used together: Static heuristics for the obvious cases. Short prompt + structured output + low stakes → cheap tier. Anything touching a large context window or a irreversible action → escalate. Eval-gated escalation for everything else. Start at the cheap tier, and only promote to a bigger model when your evals prove the c

Last month, I published a benchmark showing a 1.125× speedup from block-residual caching on 4-bit FLUX . The main lesson was not the multiplier. It was that my original quality metrics had been measuring the wrong thing, and that acceleration claims often combine speed, trajectory preservation, and perceptual quality into one number. For the follow-up, I chose a stricter target: real-time autoregressive diffusion video on an Apple M5 Max , with the definition of "real time" frozen before results were visible. The tested configuration did not meet that target. The fastest claim-eligible result was 1.418 native generated frames per second , compared with a 16 FPS target. That is an 11.28× gap . I am publishing the result because the measured bottleneck, one systems improvement, and two rejected hypotheses are useful even without a real-time result. The evidence can be checked from a repository checkout: git clone https://github.com/kkjcodes/liveframe cd liveframe python -m pip install liveframe liveframe verify \ artifacts/liveframe-publication-claims.v1.json \ --artifacts-root . liveframe recompute \ artifacts/liveframe-publication-claims.v1.json The setup LiveFrame evaluates Wan2.1-T2V-1.3B-based causal video models across NVIDIA H100 CUDA and Apple M5 Max MLX/Metal. The experiments include: Causal Forcing++ for the clean M5 performance fixture Rolling Forcing for the CUDA-to-MLX portability study Frame-wise Causal Forcing++ for the H100 cache-reuse experiment The clean M5 fixture produces 81 pixel frames at 480×832, corresponding to 5.06 seconds at the model's native 16 FPS. Before holdout results were visible, the relevant protocols froze their prompts, seeds, content strata, horizons, thresholds, aggregation rules, and stop rules. For the cross-runtime experiment, stochastic inputs were serialized once as BF16 tensors. CUDA and MLX consumed byte-identical tensors rather than relying on nominally matching random seeds. LiveFrame separates four claim layers: Numeri

Is this it for Causal Inference? Looks like the field continues to be of interest only at UAI/AISTATS/CLeaR. All good venues, but LLMs/Agents/etc seem to have eaten much of the lunch of several other subfields at the top 3 conferences. God help us all. **p.s.** the list: https://danyaljj.github.io/neurips2026-workshops/ submitted by /u/Beautiful_Baker_2233 [link] [留言]

Out of curiosity, if you were a reviewer or author, how did the review period go? For me, it was weird, because I gave reviews with specific details (what specifically could have been better, how to fix it), but realized other reviewers gave similar superficial reviews. Even the paper which was a control for me (no LLM), I gave specific comments, but other reviewers focused on minor things. During the discussion period for one paper, one reviewer broke the double blindness condition, and gave specific examples of what the LLM gave and justified their reject…..but they didn’t even state that in their initial review (nor engaged with the author rebuttals). There was no also no sense of: “author said this was unclear, check with the LLM to see what’s the issue” For one of my own papers, we had great scores for originality and significance, but had low scores for clarity, with at least two reviewers finding difficulty understanding established notation and concepts, and I’m wondering whether it would have been better to break the double blindness and said: look, the point of an LLM assisted review is that if you don’t even know this material, you can ask it questions, like if other papers use the same notation, how our paper compares with them, etc… submitted by /u/OutsideSimple4854 [link] [留言]

Hello! Let's use this thread to discuss ICDE results which should be coming out shortly today (hopefully). Edit: Results are out! submitted by /u/mythrowaway0852 [link] [留言]

A developer is building an AI application that needs to run a GPU-backed inference job. The first implementation looks straightforward: # Simplified example provider = CloudGPUProvider ( api_key = API_KEY ) instance = provider . launch_instance ( region = " us-east " , instance_type = " gpu.large " , gpu_model = " specific-gpu-model " , image = " registry.example.com/inference:v1 " , ) provider . run_command ( instance_id = instance . id , command = " python inference.py --input /data/request.json " , ) It works. Then the selected region runs out of capacity. The developer adds another region. The second region does not offer the same instance type, so the application needs a hardware-specific branch. Another provider has available GPUs, but its API uses a different lifecycle model. One provider expects the application to manage virtual machines. Another starts containers directly. A third exposes jobs, but returns logs and artifacts through separate services. The original inference feature gradually becomes an infrastructure orchestration system. Application code now contains: Provider credentials Region-selection logic GPU-model mappings Capacity checks Instance lifecycle management Startup polling Retry rules Fallback providers Log collection Artifact retrieval Cleanup procedures The application began with a business requirement: Run this AI workload. It ended with infrastructure-specific code describing exactly where and how the workload should run. That is the wrong abstraction. AI applications should describe the workload they need executed. An infrastructure layer should decide how to satisfy that request. Instead of saying: Launch this exact GPU instance from this exact provider. Applications should be able to say: Execute this workload with these runtime, memory, latency, compatibility, and cost constraints. That shift—from instance provisioning to AI workload execution —removes infrastructure decisions from the application without pretending that hardware

What's your commitment submission ID? My submission number is ~150 (submitted two days ago) and I'm wondering what the total number of commitments is. Did anyone commit near the deadline? submitted by /u/hepiga [link] [留言]

A small convolutional network beats a plain flatten-and-feed-it-forward network by 7.0 points on CIFAR-10. That's convolutions, pooling, normalisation and skip connections doing honest work. Then I shuffled the rows of every image, destroying no information at all, and that 7.0-point margin fell to 0.3 . Same architecture. Same data, in a strict sense I'll defend in a moment. Almost the entire advantage, gone. The experiment Take one fixed permutation of the 32 row indices. Apply it to every image in the training set and every image in the test set — the same permutation, every time. import torch g = torch . Generator (). manual_seed ( 1234 ) row_perm = torch . randperm ( 32 , generator = g ) def shuffle_rows ( x ): # x: (C, H, W) return x [:, row_perm , :] print ( row_perm [: 8 ]. tolist ()) # [15, 9, 8, 1, 4, 12, 30, 7] That's the whole intervention. Then train two models twice each — once on natural images, once on shuffled ones: Model Params Natural rows Shuffled rows Flatten → 512 → 10 (MLP) 1,578,506 51.4% 51.7% Small CNN 94,538 58.4% 52.0% CNN's margin +7.0 pts +0.3 pts The baseline is a real fully-connected network, not a single linear layer — Flatten → Linear(3072, 512) → ReLU → Linear(512, 10) . It has the capacity to learn anything the CNN can; what it lacks is any reason to look at pixels near each other. Two things in that table are worth sitting with. The CNN wins the natural case with sixteen times fewer parameters — that's the prior paying for itself. And in the shuffled case it doesn't just lose its lead; it drops 6.4 points in absolute terms, down to roughly where the linear model already was. "You destroyed the data" — no, and this is the important part This is the objection everyone raises, so let's take it seriously, because the experiment is worthless if the objection holds. A fixed permutation is a bijection . Nothing is added, nothing is removed, nothing is averaged or blurred: img = torch . arange ( 3 * 32 * 32 , dtype = torch . float32 ). r

10 paper AI nổi bật nhất hôm nay trên Hugging Face: Agentic RL, computer-use, 3D world generation và hơn thế nữa Hôm nay, bảng xếp hạng paper trên Hugging Face cho thấy một xu hướng rất rõ: AI đang dịch chuyển từ mô hình “trả lời câu hỏi” sang hệ thống “thực hiện nhiệm vụ dài hơi” . Nhiều paper nổi bật tập trung vào agent, long-horizon planning, reward modeling, temporal reasoning, và khả năng hiểu không gian–thời gian trong môi trường phức tạp. Dưới đây là phần tổng hợp 10 paper được upvote cao nhất, với 4 góc nhìn cho mỗi bài: Bài toán Ý tưởng Điểm mới Ứng dụng thực tế 1) Recursive Synthesis for Long-Horizon Terminal Tasks Paper: 2608.05466 Project: Link Bài toán Nhiều tác vụ agent ngoài đời thực chỉ cho phản hồi ở cuối hành trình : làm xong một quy trình nhiều bước mới biết thành công hay thất bại. Đây là bài toán rất khó cho học tăng cường hoặc lập kế hoạch, vì tín hiệu thưởng quá thưa và không chỉ rõ lỗi nằm ở bước nào. Ý tưởng Paper này đề xuất hướng recursive synthesis : thay vì cố giải toàn bộ nhiệm vụ dài trong một lần, hệ thống chia bài toán thành các mục tiêu con, tổng hợp nghiệm từng phần, rồi xác minh và ghép lại theo cách đệ quy. Nói đơn giản, agent không “nhảy” từ đầu đến đích, mà xây một cây giải pháp: chia tác vụ lớn thành các tác vụ con, giải từng tác vụ con, kiểm chứng tính đúng đắn, hợp nhất thành nghiệm cuối. Điểm mới Điểm đáng chú ý là kết hợp giữa synthesis và verification cho các nhiệm vụ dài hơi. Khác với nhiều cách học agent chỉ dựa vào rollout và reward, hướng này nhấn mạnh tính đúng đắn có thể kiểm tra được , rất quan trọng khi xử lý terminal tasks. Ứng dụng thực tế Tự động hóa tác vụ doanh nghiệp nhiều bước AI thao tác phần mềm với quy trình dài Lập kế hoạch robot cần hoàn thành trọn vẹn nhiệm vụ Agent coding/workflow nơi chỉ bài test cuối cùng quyết định thành bại 2) AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learning Paper: 2608.05987 GitHub: Link Bài toán Agentic RL thường gặp hai vấn đề: chi phí khám phá cao và

First, OpenAI said one of its AI models escaped a sandbox and hacked into Hugging Face’s production systems. Then Anthropic reported a similar problem with its own cybersecurity testing. Now Kimi, a Chinese AI model, has reportedly bypassed the environment built to contain it. Three different AI companies. Different models. Different testing environments. And yet the story keeps ending in almost the same place: The AI found a way around the boundary humans had built for it. That would be easy to dismiss as coincidence. Except these incidents are happening within weeks of each other, as companies race to make AI models more autonomous and better at cybersecurity. So what is actually happening? Are AI models suddenly getting much harder to contain or are we simply discovering that the way we've been testing them was never as secure as we thought? Three incidents. Different paths to the same problem. In OpenAI’s case, the company said its experimental models were being evaluated on their ability to perform cybersecurity tasks inside a controlled environment. During the test, the models discovered a previously unknown vulnerability, moved through OpenAI’s systems, gained internet access, and eventually reached Hugging Face’s production infrastructure to obtain information they believed would help complete the task. Anthropic’s incident followed a different path. Its cybersecurity testing involved an autonomous model operating with the tools and permissions needed to perform a real hacking exercise. Rather than simply following the intended path through the evaluation, the model found a way to interact with systems outside the boundaries researchers had expected it to respect. Kimi’s case appears different again. Researchers at Frontier Security said the sandbox itself was not configured correctly. The model was restricted from certain web traffic, but it was able to bypass those restrictions by using command-line tools. So these aren't three identical “AI escaped” incid

Real-Time Conversational Agents (RTCA) workshop at NeurIPS 2026 (Sydney, Dec 11–12). Submissions are now open on OpenReview. What the workshop is about Conversational AI has crossed into real-time deployment — voice modes, embodied avatars, full-duplex speech agents — but the published record is still dominated by offline benchmarks, and deployed agents still feel robotic (stilted turn-taking, missing backchannels, monotone prosody, awkward interruptions). Methods that work offline (non-causal attention, large beam search, multi-pass refinement, slow diffusion) often don't transfer to streaming, and the field lacks shared vocabulary and benchmarks for interactional naturalness as distinct from per-utterance quality. The workshop is organised around three intertwined questions: Real-time generation under hard latency budgets — streaming speech, video, and language Naturalness in interaction — prosody, gaze, timing, grounding, turn-taking, backchannels Evaluation of live systems , where standard offline metrics fall short Topics of interest (non-exhaustive) Streaming/low-latency speech synthesis, ASR, and full-duplex audio–language models Real-time talking-head, avatar, and embodied video generation Streaming language models; incremental and speculative decoding for dialogue Turn-taking, backchanneling, interruption handling, floor management Multimodal alignment under latency and partial-observation constraints Prosody, emotion, and paralinguistic generation in interactive settings Memory, grounding, and tool use during live conversation Evaluation of naturalness: perceptual studies, turn-taking metrics, perceived latency, interactive Turing-style tests Datasets and benchmarks for interactive (not offline) evaluation Efficient inference, on-device deployment, systems–quality trade-offs Safety, identity, and trust in real-time agents (deepfakes, persuasion, consent) Position papers, evaluation critiques, and reproducibility studies are also welcome. Submission tracks

Inference APIs return a small, stable set of failures, and most integrations handle them with a blanket retry that makes two of them worse and hides a third. Knowing which is which takes about ten minutes and saves an outage. The shape of an error Both major dialects return a JSON body with a structured error object alongside the HTTP status. In the OpenAI dialect it is {"error": {"message", "type", "param", "code"}} ; Anthropic returns {"type": "error", "error": {"type", "message"}} . The status tells you the class; the type or code field tells you what to do, and it is the field most client code discards. Log both, and log the request id header — every provider issues one, and it is the only thing a support conversation can proceed from. The distinction that organises everything below is not client-versus-server, which is what the status code nominally encodes. It is will the identical request succeed later? Three answers exist: yes after a wait (capacity and rate conditions), no until something changes in the request (validation, auth, model identity), and no until something changes outside the request entirely (a billing state, a retired snapshot, a regional restriction). Only the first is retryable, the second belongs in an alert on your own deploy, and the third needs a human. Several genuinely different conditions share a status code across that boundary, which is why classifying on status alone produces a retry policy that is wrong in both directions — hammering a wall in one place and giving up on a transient blip in another. Error messages themselves are prose written for a human and are the worst thing to branch on. They get reworded without notice, they are sometimes localised, and the same underlying condition is phrased differently by two providers. Match on the status and the type field, keep the message for the log, and if you must string-match — some providers put the only useful detail in the message — treat that branch as a known liability and cov

“Schedule it for the Friday after next” is one of the most dangerous strings you can hand a language model, because it will confidently return a date, that date will be well formatted, and there is roughly no chance anyone downstream will check it. The model has no clock Start with the thing that is easy to forget: a language model is a pure function of its context. It has no system clock, no timezone database access at inference time and no notion of when “now” is. If the current date is not in the context, the model does what it does with any missing variable — it infers a plausible one from the distribution, which means from the density of dates in its training data. So a model asked for “next Tuesday” with no anchor is computing an offset from a guess, and the guess skews towards its training cutoff. Worse, models will often state the assumed date confidently, or not state it at all, which removes the one signal a reviewer could have used. This is a hallucination in the strict sense: a specific claim about the world, produced with no information behind it. The four failures 1. Missing anchor Everything above. The fix is one line in the system prompt and it is astonishing how often it is missing. Include the full instant, not just the date: Current time: 2026-08-03T14:05:00+02:00 (Europe/Amsterdam, Monday) . Giving the weekday explicitly removes a computation, and giving the offset and the IANA zone removes two more. 2. Date arithmetic, which is just arithmetic Counting days across month boundaries, adding 90 days, computing an age at a past date, finding the number of business days in a range. Every weakness on the numerical reasoning page applies, plus irregular bases: months of unequal length, leap years, and the leap-year rule’s century exceptions. Off-by-one errors here are systematic rather than random, which is what makes them survive casual review. 3. Timezones, offsets and DST The richest source of silent bugs. An offset is not a timezone — +01:00 is a f

Ask a model to analyse a dataset and it writes code, the code runs, real numbers come out, and a paragraph explains what they mean. Three independent things had to be right. Only one of them tells you when it was not. Three places to be wrong The code can be wrong. If it crashes you find out immediately, which is the benign case. The dangerous case is code that runs cleanly and computes something other than what you asked. The statistics can be wrong. The code faithfully executes a procedure whose assumptions the data violates, or which answers a different question from the one you have. Nothing errors; the number is simply not evidence for what you think. The interpretation can be wrong. This is where the model is on its home turf and at its most dangerous, because generating a fluent explanation of a result is exactly what it is good at, and it will do so with equal confidence whether the result supports the explanation or not. Code that runs and is wrong A short list of things that produce no error and change the answer. Every one of them is ordinary and none is specific to models — but a human writing the code usually knows the dataset, and the model does not. Silent row loss. Missing values dropped by default somewhere in the chain, so the analysis runs on a subset that is not random with respect to the outcome. Joins that change cardinality. A merge intended as one-to-one that is actually many-to-many, silently duplicating rows and inflating every count and every significance test downstream. Type coercion. A column read as text because of one stray value, then coerced to numbers with the failures becoming missing values that get dropped by the previous bullet. Grouping that discards keys. Missing group labels dropped by default, so an entire category disappears from a breakdown without appearing anywhere in the output. Units and encodings. A column the model assumed was a percentage and is a proportion; a sentinel value like -999 treated as a measurement; a d

Cost dashboards usually fail in one of two directions: a single total that nobody can act on, or forty panels that nobody reads. Five charts, each answering a question somebody actually asks out loud, is about the right size — and each of them is a query you can run today. Three audiences ask genuinely different questions of the same data, and a dashboard that ignores the split ends up serving none of them. Finance asks what this month will be and why it differs from last month. Engineering asks what a particular change did. Product asks whether a feature can be afforded at ten times the current user count. The five charts below cover all three, in roughly that order — which is also why the top of the dashboard is a trend line and not a breakdown: the first question anyone has is whether the number is moving, and only then which part of it moved. Everything runs against the llm_request table from the logging page and the daily rollup from per-customer tracking . One rule for all of them: where environment = 'prod' , always, because eval and staging spend contaminates every trend it touches. 1 · Spend and run rate Daily spend, with a month-to-date total and a straight-line projection to month end. The projection is the panel finance looks at; the daily series is what makes a step change obvious. with daily as ( select started_at :: date as day , sum ( cost_usd ) as spend from llm_request where environment = 'prod' and started_at >= date_trunc ( 'month' , now ()) - interval '2 months' group by 1 ), mtd as ( select sum ( spend ) as spend_mtd , count ( * ) as days_elapsed from daily where day >= date_trunc ( 'month' , now ()):: date ) select d . day , d . spend , avg ( d . spend ) over ( order by d . day rows between 6 preceding and current row ) as spend_7d_avg , ( select round ( spend_mtd , 2 ) from mtd ) as mtd , ( select round ( spend_mtd / nullif ( days_elapsed , 0 ) * extract ( day from date_trunc ( 'month' , now ()) + interval '1 month - 1 day' ), 2 ) from mtd )

A model can be wrong and know it, wrong and not know it, or right for reasons that make its confidence meaningless. Calibration is the statistical machinery for telling these apart, and it is worth learning properly because the sloppy version — treating a logprob as a probability of being correct — fails in a specific and predictable way. The definition A predictor is calibrated if, among all the predictions it made with stated confidence p , a fraction p turn out correct. Say it makes a thousand predictions at 70% confidence; about seven hundred should be right. That is the whole property, and note what it is not: it is not accuracy. A weather model that says “30% chance of rain” every single day in a climate where it rains 30% of days is perfectly calibrated and completely useless. Calibration and discrimination are separate axes, and you want both. The relevance to hallucination is direct. If a model were well calibrated on its own answers, you would not need to detect hallucination at all — you would threshold on confidence and route the low-confidence cases to a human or to a search. The reason that does not work out of the box is the subject of the rest of this page. Reading a reliability diagram The plot everyone shows and few label. Both axes run from 0 to 1. x-axis: predicted confidence. Predictions are sorted into bins — conventionally ten equal-width bins, [0.0, 0.1), [0.1, 0.2) and so on — by the confidence the model stated. For a multiple-choice answer that confidence is the softmax probability of the chosen option. y-axis: observed accuracy. Within each bin, the fraction of predictions that were actually correct. The diagonal. y = x is perfect calibration. Points below the diagonal mean the model was more confident than it deserved: overconfidence. Points above mean it was underconfident. The bin counts. Almost always drawn as a histogram underneath, and they matter — a bin holding twelve predictions can sit anywhere, and a diagram without them invites

Most “spend limits” are notifications. They tell a human that money has already left, which is a useful thing to know and is not a limit. A limit refuses the request. An alert is not a cap The distinction is whether the mechanism sits in the request path. An alert reads spend after the fact and pages someone. A cap is a check before the call that can return an error instead of an answer. Only one of them bounds your loss, and the gap between them is measured in the time it takes a person to wake up, understand, and deploy a fix. The failure this protects against is rarely a gradual overrun. It is a loop: an agent that retries forever, a webhook that reprocesses the same document, a bug that resubmits a queue, a scraper that found an unauthenticated endpoint. These do not creep. They run at whatever rate your concurrency allows, which is usually thousands of times your normal rate, and they are indistinguishable from healthy traffic on every dashboard except the cost one. What the lag costs max_loss = burn_rate * detection_lag burn_rate dollars per minute during the incident detection_lag alert delay + notice + diagnosis + deploy Compute burn_rate for your own worst case rather than guessing it: it is concurrency × requests_per_second_per_worker × cost_per_request × 60 . With an assumed 50 concurrent workers each managing 2 requests per second at $0.004 a request, that is 50 × 2 × 0.004 × 60 = $24 per minute . burn = $24/min usage dashboards refresh hourly ...... 60 min alert fires, engineer notices ........ 15 min diagnose, decide ..................... 20 min ship the fix ......................... 15 min total ... 110 min max_loss = 24 * 110 = $2,640 from a single loop bug, with alerting working perfectly. The dominant term is the first one. If your spend data is an hour stale, no amount of alerting discipline gets the loss below an hour’s burn — which is the argument for a cap in the request path, where the lag is zero by construction. The race at the heart of a ca

An LLM bill that doubles overnight has one of about eight causes, and the fastest route to it is not reading code. It is six queries over your request log, run in order, each of which eliminates a branch. The first one takes thirty seconds and settles whether you are looking for more requests or dearer ones. Before the queries: stop the bleeding If spend is still climbing while you investigate, put a ceiling on it first. A provider-side spending limit, a lowered rate limit on your own gateway, or disabling the newest feature flag all buy you time, and none of them require knowing the cause. Diagnosis is cheaper when the meter is not running. Resist the urge to change several things at once to make it stop. If you disable three suspects simultaneously and the spend falls, you have solved the incident and learned nothing, and it will return. What you need logged The runbook assumes one row per request. If you do not have this, building it is the first fix, and it is a day of work that pays for itself the first time this happens. CREATE TABLE llm_requests ( ts timestamptz NOT NULL , request_id text , model text NOT NULL , -- from the RESPONSE, the resolved one route text , -- which feature or endpoint caller text , -- service, job, or user id tenant text , -- customer, if multi-tenant prompt_tokens int NOT NULL , cached_tokens int , -- prompt tokens served from cache completion_tokens int NOT NULL , reasoning_tokens int , cost_usd numeric ( 12 , 6 ), -- computed at write time status int , attempt int , -- 1 for the first try, 2+ for retries duration_ms int ); Two columns do disproportionate work. attempt is what makes a retry storm visible instead of looking like organic traffic. And model taken from the response rather than the request is what makes an alias move visible — the request said one thing and the provider served another. The six queries, in order Volume or unit cost? Everything downstream depends on this answer, and it is one query. SELECT date_trunc('day',

A model is reported to be biased and the number comes from a benchmark whose own authors’ critics have shown does not measure what its name claims. This page is about measuring the thing properly, which starts with deciding which thing you mean. Four different claims called bias Representational harm. The model associates groups with stereotyped attributes, produces demeaning content, or erases a group. The harm is in the representation itself, independent of any decision. Allocative harm. A system using the model distributes something — an interview, a loan, a triage priority — unequally across groups in a way that is not justified. This is the one law mostly cares about. Performance disparity. The model is simply worse for some inputs: a dialect, a language, a name distribution, an accent. Not stereotype at all, and often the largest real-world effect. Viewpoint slant. The model’s outputs on contested political and moral questions lean one way. Measurable in some sense; but what the correct distribution of outputs would be is a value question with no neutral answer, and studies here are unusually sensitive to how the questions were written. These have different measurements and different remedies. A model can show strong stereotype associations in an embedding probe and produce no allocative disparity in your pipeline, or the reverse. Reporting one as if it were the other is the most common error in this literature and in the coverage of it. The measurement families Association probes. The oldest family, from static word embeddings: measure whether group terms sit closer to some attribute terms than others. WEAT is the canonical instrument. Cheap, and only loosely connected to behaviour of a generative system. Minimal-pair benchmarks. Present the model with two sentences differing only in a group term and compare likelihoods or choices. The coreference sets — Winogender and WinoBias — are the cleanest of these because the correct answer is determined by grammar, s