标签：#learn

If you work with measurements often enough, you eventually run into the same problem: the value you have isn't in the unit you need. A product specification might be in inches. A construction drawing might use feet. A European supplier might give you dimensions in centimeters or millimeters. The actual formulas are usually simple. Finding the right conversion, avoiding rounding mistakes, and checking a large list of values can be more annoying than the math itself. Here are the conversions I use most often and a few practical ways to work with them. Inches to centimeters The basic relationship is: 1 inch = 2.54 centimeters So the formula is: centimeters = inches × 2.54 For example: 10 inches × 2.54 = 25.4 cm This is probably the most common conversion when moving between imperial and metric measurements. If you just need to check a value quickly, Pulgadas a CM has an interactive converter along with a conversion table and frequently asked questions. Centimeters to inches Going in the opposite direction means dividing by 2.54: inches = centimeters ÷ 2.54 For example: 25.4 cm ÷ 2.54 = 10 inches You can use the CM a Pulgadas converter when you need to work in this direction. This is particularly useful when a measurement is provided in centimeters but the product, tool, or specification you're working with uses inches. Meters to inches Meters are larger units, so the conversion factor is correspondingly larger. One meter contains approximately: 39.3700787 inches Therefore: inches = meters × 39.3700787 For example: 2 meters ≈ 78.7401574 inches For a quick calculation, you can use the Metros a Pulgadas converter . This conversion can come up when working with room dimensions, furniture measurements, fabric, sports equipment, or other products where metric and imperial specifications are mixed. Inches to meters The reverse calculation is: meters = inches × 0.0254 For example: 100 inches × 0.0254 = 2.54 meters The Pulgadas a Metros converter is useful when an imperial meas

Inference APIs return a small, stable set of failures, and most integrations handle them with a blanket retry that makes two of them worse and hides a third. Knowing which is which takes about ten minutes and saves an outage. The shape of an error Both major dialects return a JSON body with a structured error object alongside the HTTP status. In the OpenAI dialect it is {"error": {"message", "type", "param", "code"}} ; Anthropic returns {"type": "error", "error": {"type", "message"}} . The status tells you the class; the type or code field tells you what to do, and it is the field most client code discards. Log both, and log the request id header — every provider issues one, and it is the only thing a support conversation can proceed from. The distinction that organises everything below is not client-versus-server, which is what the status code nominally encodes. It is will the identical request succeed later? Three answers exist: yes after a wait (capacity and rate conditions), no until something changes in the request (validation, auth, model identity), and no until something changes outside the request entirely (a billing state, a retired snapshot, a regional restriction). Only the first is retryable, the second belongs in an alert on your own deploy, and the third needs a human. Several genuinely different conditions share a status code across that boundary, which is why classifying on status alone produces a retry policy that is wrong in both directions — hammering a wall in one place and giving up on a transient blip in another. Error messages themselves are prose written for a human and are the worst thing to branch on. They get reworded without notice, they are sometimes localised, and the same underlying condition is phrased differently by two providers. Match on the status and the type field, keep the message for the log, and if you must string-match — some providers put the only useful detail in the message — treat that branch as a known liability and cov

“Schedule it for the Friday after next” is one of the most dangerous strings you can hand a language model, because it will confidently return a date, that date will be well formatted, and there is roughly no chance anyone downstream will check it. The model has no clock Start with the thing that is easy to forget: a language model is a pure function of its context. It has no system clock, no timezone database access at inference time and no notion of when “now” is. If the current date is not in the context, the model does what it does with any missing variable — it infers a plausible one from the distribution, which means from the density of dates in its training data. So a model asked for “next Tuesday” with no anchor is computing an offset from a guess, and the guess skews towards its training cutoff. Worse, models will often state the assumed date confidently, or not state it at all, which removes the one signal a reviewer could have used. This is a hallucination in the strict sense: a specific claim about the world, produced with no information behind it. The four failures 1. Missing anchor Everything above. The fix is one line in the system prompt and it is astonishing how often it is missing. Include the full instant, not just the date: Current time: 2026-08-03T14:05:00+02:00 (Europe/Amsterdam, Monday) . Giving the weekday explicitly removes a computation, and giving the offset and the IANA zone removes two more. 2. Date arithmetic, which is just arithmetic Counting days across month boundaries, adding 90 days, computing an age at a past date, finding the number of business days in a range. Every weakness on the numerical reasoning page applies, plus irregular bases: months of unequal length, leap years, and the leap-year rule’s century exceptions. Off-by-one errors here are systematic rather than random, which is what makes them survive casual review. 3. Timezones, offsets and DST The richest source of silent bugs. An offset is not a timezone — +01:00 is a f

Ask a model to analyse a dataset and it writes code, the code runs, real numbers come out, and a paragraph explains what they mean. Three independent things had to be right. Only one of them tells you when it was not. Three places to be wrong The code can be wrong. If it crashes you find out immediately, which is the benign case. The dangerous case is code that runs cleanly and computes something other than what you asked. The statistics can be wrong. The code faithfully executes a procedure whose assumptions the data violates, or which answers a different question from the one you have. Nothing errors; the number is simply not evidence for what you think. The interpretation can be wrong. This is where the model is on its home turf and at its most dangerous, because generating a fluent explanation of a result is exactly what it is good at, and it will do so with equal confidence whether the result supports the explanation or not. Code that runs and is wrong A short list of things that produce no error and change the answer. Every one of them is ordinary and none is specific to models — but a human writing the code usually knows the dataset, and the model does not. Silent row loss. Missing values dropped by default somewhere in the chain, so the analysis runs on a subset that is not random with respect to the outcome. Joins that change cardinality. A merge intended as one-to-one that is actually many-to-many, silently duplicating rows and inflating every count and every significance test downstream. Type coercion. A column read as text because of one stray value, then coerced to numbers with the failures becoming missing values that get dropped by the previous bullet. Grouping that discards keys. Missing group labels dropped by default, so an entire category disappears from a breakdown without appearing anywhere in the output. Units and encodings. A column the model assumed was a percentage and is a proportion; a sentinel value like -999 treated as a measurement; a d

Cost dashboards usually fail in one of two directions: a single total that nobody can act on, or forty panels that nobody reads. Five charts, each answering a question somebody actually asks out loud, is about the right size — and each of them is a query you can run today. Three audiences ask genuinely different questions of the same data, and a dashboard that ignores the split ends up serving none of them. Finance asks what this month will be and why it differs from last month. Engineering asks what a particular change did. Product asks whether a feature can be afforded at ten times the current user count. The five charts below cover all three, in roughly that order — which is also why the top of the dashboard is a trend line and not a breakdown: the first question anyone has is whether the number is moving, and only then which part of it moved. Everything runs against the llm_request table from the logging page and the daily rollup from per-customer tracking . One rule for all of them: where environment = 'prod' , always, because eval and staging spend contaminates every trend it touches. 1 · Spend and run rate Daily spend, with a month-to-date total and a straight-line projection to month end. The projection is the panel finance looks at; the daily series is what makes a step change obvious. with daily as ( select started_at :: date as day , sum ( cost_usd ) as spend from llm_request where environment = 'prod' and started_at >= date_trunc ( 'month' , now ()) - interval '2 months' group by 1 ), mtd as ( select sum ( spend ) as spend_mtd , count ( * ) as days_elapsed from daily where day >= date_trunc ( 'month' , now ()):: date ) select d . day , d . spend , avg ( d . spend ) over ( order by d . day rows between 6 preceding and current row ) as spend_7d_avg , ( select round ( spend_mtd , 2 ) from mtd ) as mtd , ( select round ( spend_mtd / nullif ( days_elapsed , 0 ) * extract ( day from date_trunc ( 'month' , now ()) + interval '1 month - 1 day' ), 2 ) from mtd )

A model can be wrong and know it, wrong and not know it, or right for reasons that make its confidence meaningless. Calibration is the statistical machinery for telling these apart, and it is worth learning properly because the sloppy version — treating a logprob as a probability of being correct — fails in a specific and predictable way. The definition A predictor is calibrated if, among all the predictions it made with stated confidence p , a fraction p turn out correct. Say it makes a thousand predictions at 70% confidence; about seven hundred should be right. That is the whole property, and note what it is not: it is not accuracy. A weather model that says “30% chance of rain” every single day in a climate where it rains 30% of days is perfectly calibrated and completely useless. Calibration and discrimination are separate axes, and you want both. The relevance to hallucination is direct. If a model were well calibrated on its own answers, you would not need to detect hallucination at all — you would threshold on confidence and route the low-confidence cases to a human or to a search. The reason that does not work out of the box is the subject of the rest of this page. Reading a reliability diagram The plot everyone shows and few label. Both axes run from 0 to 1. x-axis: predicted confidence. Predictions are sorted into bins — conventionally ten equal-width bins, [0.0, 0.1), [0.1, 0.2) and so on — by the confidence the model stated. For a multiple-choice answer that confidence is the softmax probability of the chosen option. y-axis: observed accuracy. Within each bin, the fraction of predictions that were actually correct. The diagonal. y = x is perfect calibration. Points below the diagonal mean the model was more confident than it deserved: overconfidence. Points above mean it was underconfident. The bin counts. Almost always drawn as a histogram underneath, and they matter — a bin holding twelve predictions can sit anywhere, and a diagram without them invites

Most “spend limits” are notifications. They tell a human that money has already left, which is a useful thing to know and is not a limit. A limit refuses the request. An alert is not a cap The distinction is whether the mechanism sits in the request path. An alert reads spend after the fact and pages someone. A cap is a check before the call that can return an error instead of an answer. Only one of them bounds your loss, and the gap between them is measured in the time it takes a person to wake up, understand, and deploy a fix. The failure this protects against is rarely a gradual overrun. It is a loop: an agent that retries forever, a webhook that reprocesses the same document, a bug that resubmits a queue, a scraper that found an unauthenticated endpoint. These do not creep. They run at whatever rate your concurrency allows, which is usually thousands of times your normal rate, and they are indistinguishable from healthy traffic on every dashboard except the cost one. What the lag costs max_loss = burn_rate * detection_lag burn_rate dollars per minute during the incident detection_lag alert delay + notice + diagnosis + deploy Compute burn_rate for your own worst case rather than guessing it: it is concurrency × requests_per_second_per_worker × cost_per_request × 60 . With an assumed 50 concurrent workers each managing 2 requests per second at $0.004 a request, that is 50 × 2 × 0.004 × 60 = $24 per minute . burn = $24/min usage dashboards refresh hourly ...... 60 min alert fires, engineer notices ........ 15 min diagnose, decide ..................... 20 min ship the fix ......................... 15 min total ... 110 min max_loss = 24 * 110 = $2,640 from a single loop bug, with alerting working perfectly. The dominant term is the first one. If your spend data is an hour stale, no amount of alerting discipline gets the loss below an hour’s burn — which is the argument for a cap in the request path, where the lag is zero by construction. The race at the heart of a ca

An LLM bill that doubles overnight has one of about eight causes, and the fastest route to it is not reading code. It is six queries over your request log, run in order, each of which eliminates a branch. The first one takes thirty seconds and settles whether you are looking for more requests or dearer ones. Before the queries: stop the bleeding If spend is still climbing while you investigate, put a ceiling on it first. A provider-side spending limit, a lowered rate limit on your own gateway, or disabling the newest feature flag all buy you time, and none of them require knowing the cause. Diagnosis is cheaper when the meter is not running. Resist the urge to change several things at once to make it stop. If you disable three suspects simultaneously and the spend falls, you have solved the incident and learned nothing, and it will return. What you need logged The runbook assumes one row per request. If you do not have this, building it is the first fix, and it is a day of work that pays for itself the first time this happens. CREATE TABLE llm_requests ( ts timestamptz NOT NULL , request_id text , model text NOT NULL , -- from the RESPONSE, the resolved one route text , -- which feature or endpoint caller text , -- service, job, or user id tenant text , -- customer, if multi-tenant prompt_tokens int NOT NULL , cached_tokens int , -- prompt tokens served from cache completion_tokens int NOT NULL , reasoning_tokens int , cost_usd numeric ( 12 , 6 ), -- computed at write time status int , attempt int , -- 1 for the first try, 2+ for retries duration_ms int ); Two columns do disproportionate work. attempt is what makes a retry storm visible instead of looking like organic traffic. And model taken from the response rather than the request is what makes an alias move visible — the request said one thing and the provider served another. The six queries, in order Volume or unit cost? Everything downstream depends on this answer, and it is one query. SELECT date_trunc('day',

A model is reported to be biased and the number comes from a benchmark whose own authors’ critics have shown does not measure what its name claims. This page is about measuring the thing properly, which starts with deciding which thing you mean. Four different claims called bias Representational harm. The model associates groups with stereotyped attributes, produces demeaning content, or erases a group. The harm is in the representation itself, independent of any decision. Allocative harm. A system using the model distributes something — an interview, a loan, a triage priority — unequally across groups in a way that is not justified. This is the one law mostly cares about. Performance disparity. The model is simply worse for some inputs: a dialect, a language, a name distribution, an accent. Not stereotype at all, and often the largest real-world effect. Viewpoint slant. The model’s outputs on contested political and moral questions lean one way. Measurable in some sense; but what the correct distribution of outputs would be is a value question with no neutral answer, and studies here are unusually sensitive to how the questions were written. These have different measurements and different remedies. A model can show strong stereotype associations in an embedding probe and produce no allocative disparity in your pipeline, or the reverse. Reporting one as if it were the other is the most common error in this literature and in the coverage of it. The measurement families Association probes. The oldest family, from static word embeddings: measure whether group terms sit closer to some attribute terms than others. WEAT is the canonical instrument. Cheap, and only loosely connected to behaviour of a generative system. Minimal-pair benchmarks. Present the model with two sentences differing only in a group term and compare likelihoods or choices. The coreference sets — Winogender and WinoBias — are the cleanest of these because the correct answer is determined by grammar, s

Using a model to grade another model’s output is the only approach that scales to open-ended text. It is also the point at which your measurement device becomes a second stochastic system with opinions, and the difference between a useful judge and a number-generator is entirely in whether you validated it. A judge is an instrument, not an oracle Think of the judge the way a lab thinks about a thermometer. It has a reading, a bias, a precision, and a range over which it is trustworthy — and none of those are known until you check it against a reference. The reference is human labels. There is no way around this: a judge whose agreement with humans on your task is unknown produces numbers whose meaning is unknown, however many decimal places the harness prints. The good news is that the calibration is a one-off cost of a few hundred human labels, after which the judge runs for essentially free on every subsequent evaluation. That trade is what makes judges worth the trouble. What the published agreement figures say The standard reference is Zheng et al., 2023, “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena”. On their setup, a strong judge model agreed with human expert preferences at a rate above 80% — which the authors note is comparable to the agreement rate between two human experts on the same comparisons. That framing is the important part: the ceiling for a judge is not perfect agreement, it is human-human agreement, because the humans disagree with each other on genuinely ambiguous items. The same paper documents the failure modes that come with it — position bias, verbosity bias, self-enhancement bias, and weakness on maths and reasoning items where the judge must itself solve the problem to grade it. So the honest summary of the literature is: a well-constructed judge on general chat quality can approach human-level agreement, and it does so while carrying systematic biases that you have to design around. It is not evidence that your judge, on your

You ask for every line item on the invoice. There are ten. You get seven, the JSON validates, and nothing anywhere reports a problem. This is the single most reported structured-output bug and at least half the time the model is not the cause. Four causes, wildly different fixes Cause Description Truncation finish_reason == 'length'. The list was cut off mid-flight. Your max_tokens, not the model's recall. Chunk boundary Items 8-10 were on a page you did not send. Check what text actually reached the model. Dropped constraint You set minItems and the provider ignored it, so nothing enforced anything. Genuine omission Everything was present and the model stopped early. The only one that is actually about the model. Diagnose in that order, because the first three are cheap to rule out and the fourth is the expensive one to work on. Log finish_reason , usage.completion_tokens and the length of the text you sent on every extraction call and the first two answer themselves. The constraint you thought you set minItems and maxItems sit outside the documented supported keyword set for hosted strict modes. Depending on the stack, sending them either gets you a 400 naming the keyword — fine, you learn immediately — or a 200 where the keyword was quietly discarded. The second case is the trap, because your schema is now a comment. You believe a floor is enforced, the API returned success, and the array is short. Nothing in any log says the constraint was never applied. Find out which of the two your endpoint does before you rely on it — and either way, put “every item, do not summarise or skip” in the array’s description , since that reaches the model whether or not the keyword survives. Why counting is hard for a decoder There is no counter. Each token is produced from the context, and the context contains the items already emitted — so “have I got them all” is not a lookup, it is a judgement the model re-makes at every array element from what it can see. Two structural conse

An inference key is a payment instrument with an API. That is the property that makes it different from most credentials you manage: the person who steals it does not need your data to profit, because the key itself buys something they want. What makes an inference key different It is directly monetisable. A stolen database credential needs a buyer for the data. A stolen inference key is resold as capacity within hours, and automated scanners harvest public repositories continuously looking for exactly this. The loss accrues while you sleep. Usage-based billing means the damage is a function of elapsed time and rate limit, not of a single event. This is the argument for hard caps over careful monitoring. It is passed around more than most secrets. Notebooks, evaluation scripts, a colleague’s laptop, a CI job, an agent’s own environment. Every one of those is a copy you do not control. The blast radius is often the whole account. Where a provider offers one key with full access, a leak is total. Where it offers scoped keys, use them — this is the single biggest lever available. Leak paths specific to AI applications Generic advice — do not commit secrets, use a manager — is correct and widely published. These are the paths that only exist because there is a model in the system, and they are the ones that survive a conventional review: The key in the context. A key pasted into a system prompt so a tool “has access to it”. It is now one paraphrase from the transcript, and the transcript is stored. Traces and observability. LLM tracing tools capture full request bodies by default. If a header, a tool argument or an environment dump ends up in a span, your key is in a third-party dashboard with a broader access list than your secret manager. Prompt and response logs. Same problem, your own infrastructure. A logger that prints the request object on error will print the Authorization header. Evaluation datasets. Captured production traffic reused as an eval set, then share

Most LLM alerting starts as a threshold on latency and a threshold on error rate, fires nine times in the first week, and is muted by the second. The fix is not better thresholds. It is a different trigger model and a much shorter list of things allowed to page. Level-triggered, not edge-triggered An edge-triggered alert fires on a transition: latency crossed 3 seconds, error rate spiked. It is easy to write and it is why your phone buzzed at 03:00 about a condition that resolved itself in forty seconds. A level-triggered alert asks a different question — is the system currently in a bad state, and has it been for long enough to matter? Concretely, the difference is that the alert condition is evaluated over a window and describes a sustained state, and it clears when the state clears rather than when someone acknowledges it. Every rule below is of that shape. Anything that fires on a single scrape does not belong in a paging policy; put it in a dashboard. Page on symptoms, ticket on causes The reliable partition, straight out of ordinary SRE practice and entirely applicable here: Page when users are being harmed now, and a human can do something about it in minutes. That is a small list: the feature is failing, the feature is unusably slow, or money is leaving the building at an unplanned rate. Ticket when something is degraded, trending wrong, or will bite in days. Rising retry rate. One provider slower than usual while failover is absorbing it. Attribution coverage slipping. Neither for everything else. If nobody would act on it, it is a chart. The distinction matters more for LLM features than for a normal service because so many of the interesting signals are causes : a provider 429 rate, a fallback rate, a cache-hit drop. If failover is working, none of those are user-visible and none of them should wake anyone. They are exactly what you want in the morning ticket queue. Burn-rate alerts, with the numbers The standard design — described in Google’s Site Reliab

Most discussion of AI in peer review argues about whether the reviews are any good. That is the second question. The first one is that a manuscript under review is somebody else’s confidential unpublished work, and pasting it into a service is a disclosure you were not entitled to make. The argument that comes first When you accept a review invitation you accept a confidentiality undertaking. The manuscript is unpublished, it usually contains results the authors have not yet established priority on, and in the case of grant review it contains an unfunded research plan — arguably the most commercially and academically sensitive document in the whole system. You agreed not to share it. Sending it to a third-party service is sharing it. That is true whether or not the provider trains on it, whether or not it is retained, and whether or not anyone ever reads it. The undertaking was not “do not let this be trained on”; it was “do not disclose this”, and transmission to a party the authors never agreed to is disclosure. Retention and training policies affect how bad the breach is, not whether one occurred. Notice what this argument does not depend on. Not model quality, not hallucination, not bias. It would apply identically to a perfect system, which is why it is the argument that has actually driven policy, and why it will not be resolved by better models. It can only be resolved by changing where the computation happens — a model running on infrastructure already covered by the confidentiality arrangement raises a different question from a consumer chat interface, and any serious policy will distinguish them. The second argument: accountability A review is a named expert’s judgement. Its value to an editor is not the prose; it is that a person who knows the field read the paper and formed a view they are willing to stand behind. Generated text can simulate the prose and cannot supply the judgement. Editors describe the resulting artefact recognisably: fluent, correctly

Two quite different things are discussed under one heading, and almost all the confusion comes from that. One is a researcher using a model to draft, edit or translate work they did. The other is fabricated content submitted to inflate a publication record. The first is a disclosure question. The second is fraud, and it is not new. Two problems wearing one name A non-native English speaker using a model to make their methods section readable has done nothing wrong and has improved the literature. A paper mill generating plausible manuscripts at volume has committed fraud, and would have done so with or without a language model — mills existed, using image manipulation, template text and fabricated data, long before this technology arrived. Keeping them apart matters because they call for opposite responses. The first needs a disclosure norm and nothing else. The second needs content verification, and content verification does not care what tool produced the content. Any policy built around detecting machine text will punish the first group and miss most of the second, because fabricated research that has been lightly rewritten is indistinguishable from careful assisted writing. It is also worth being clear about where the demand comes from, because it explains why no technical measure will resolve this. Paper mills exist because publication counts are used as a proxy for research contribution in hiring, promotion and institutional ranking, in systems large enough that buying an authorship is a rational purchase for some buyers. Generative tools lowered the cost of supplying that demand; they did not create it. A detector, even a perfect one, sits downstream of an incentive that would simply route around it — which is why the interventions with the best track record are the ones that attack verifiability, such as requiring data and code, rather than the ones that attack production. What the artefacts look like Leftover interface text. Phrases that belong to a chat in

Whether your clinical software is a regulated medical device is decided by what you claim it does, not by how it is built. The same model can be an unregulated administrative tool with one intended-use statement and a class III device with another, and the statement is yours to write. Information, not legal advice, and not clinical or regulatory advice. Reviewed 4 August 2026. Device classification is fact-specific and the consequences of getting it wrong include enforcement action and product withdrawal. Use a regulatory professional. This page describes the questions that decide the answer; it does not answer them for your product. The line, in one paragraph per jurisdiction European Union. Software is a medical device if the manufacturer intends it for a medical purpose — diagnosis, prevention, monitoring, prediction, prognosis, treatment or alleviation of disease — as set out in the definition in Regulation (EU) 2017/745, the Medical Device Regulation, or the corresponding definition in the In Vitro Diagnostic Regulation (EU) 2017/746 where it works on specimens. There is no clinical decision support carve-out. Software that drives or influences the use of a device, or provides information used to take decisions for diagnostic or therapeutic purposes, is in. United States. Software is a device under the Federal Food, Drug, and Cosmetic Act if it is intended for use in the diagnosis, cure, mitigation, treatment or prevention of disease — but section 520(o), added by the 21st Century Cures Act in 2016, excludes certain clinical decision support software from the device definition entirely, on four cumulative conditions. That carve-out has no EU equivalent and it is the single biggest structural difference between the two regimes. EU: qualification then classification Two questions in order. Qualification asks whether it is a device at all. Classification asks which class, which determines the conformity assessment route and whether a notified body is involved. Qua

One rule settles nearly every case: a model may help you say what is true about you, and it may not decide what is true about you. Drafting is help. Supplying the content of a claim about your own experience is not. The line, and why it is there An application is a set of representations about a person, made by that person, on which somebody else will rely. That is what makes fabrication in one different in kind from fabrication in an essay: there is a party who acts on it, and there are consequences downstream for colleagues, clients and sometimes patients. So the test is not “did a machine touch this”. It is “does the document assert something the applicant does not know to be true”. A cover letter drafted from your notes and edited by you asserts nothing you did not supply. A cover letter that describes a project you did not run asserts something false regardless of who typed it, and would be equally dishonest written by a friend. Case Description rewriting: fine Rewriting your own bullet points more clearly. Fixing grammar. Translating your industry's jargon into the target industry's. Cutting 900 words to 300. Generating ten possible openings so you can choose one. decoding: fine Asking what a job advert is actually asking for, then checking your own experience against that list yourself. metrics: not fine Letting it fill in achievements, metrics or responsibilities you have not verified. 'Increased conversion by 32%' is a fact about the world; if you do not know the number, it is a fabrication with a number in it. motivation: not fine Any statement of motivation you have not read and would not say out loud. 'I have long admired your work in X' when you have not is a small lie that is very cheap to expose in an interview. assessments: not fine Completing an assessment designed to measure your unaided ability, where the employer has said not to, or where the whole point of the task is the thing you outsourced. What genuinely helps The honest uses are also the ef

Automatic top-up — the feature that charges a saved card when a customer’s balance falls below their threshold — could never have succeeded for anybody. The invoice was constructed in the wrong currency, and every attempt would have failed in a way that told the customer their card was bad. This is the whole write-up, in the shape we would want any incident written in. Summary An invoice does not take its currency from the line items attached to it. It takes it from the customer’s default currency, or failing that from the Stripe account’s — which is EUR for a Dutch business. Every price in this product is denominated in USD. Finalising the invoice therefore failed with a currency-conflict error, on every automatic top-up, unconditionally. The manual top-up path was never affected, because a Checkout Session takes its currency from the first line item rather than from the customer record. That difference is why the bug could exist in a product whose payment flow demonstrably worked. Impact Dimension Description Customers affected None. The defect was found before the path carried real traffic. This is stated plainly rather than omitted, because a postmortem that lets a near miss read as an outage is as dishonest as one that hides an outage. What would have happened Every automatic top-up fails. The failure surfaces as a payment error, which the failure counter records as a strike, and after three strikes the customer's automatic top-up is switched off entirely. What the customer would have concluded That their card was declined. The message they receive says the saved card could not be charged. They would have gone and fixed a card that was working perfectly. Secondary effect A customer relying on automatic top-up to keep a production integration serving would have run out of credit silently, at whatever hour their traffic happened to cross the threshold. The second and third rows are what make this worth writing up. A defect that fails loudly and correctly is a bug

“AI discovered a new material.” “AI found a drug candidate.” “AI solved protein folding.” Each of those sentences can be true, badly misleading, or flatly wrong depending on one thing the sentence does not tell you: how far the result got from the model before somebody wrote it down. The sentence that hides four different claims Take a single headline: a model proposed a molecule that binds a protein implicated in a disease. That sentence is compatible with at least four very different states of the world. The molecule might exist only as a string in a file. It might have been synthesised. It might have bound the protein in a test tube. Or it might have improved an outcome in a person. Those four are separated by years, by orders of magnitude in cost, and by a probability of success that drops at every step — and press coverage routinely reports the first as though it were the fourth. This is not a complaint about journalism. It is the single most useful thing to internalise about the whole field, because once you have the ladder in your head you can grade a claim in about ten seconds, and you can do it for a subject you know nothing about. The ladder The rungs are the same in every discipline. Only the names of the instruments change. Rung Description 1 · Output The model emitted something: a structure, a score, a candidate, a forecast. Nothing has been checked. Everything downstream is conditional on this being worth checking. 2 · Retrospective The output was compared against data that already existed — held-out structures, historical weather, known compounds. This is where nearly all published numbers live, and it is entirely dependent on the held-out set resembling the future. 3 · Prospective The prediction was made first and the answer arrived afterwards. A forecast verified against what the weather then did. A candidate synthesised after being proposed. This rung is qualitatively stronger than rung 2 and much rarer. 4 · Confirmed An independent method establis

Every team already knows how to put a feature behind a flag. What is different here is that the thing most likely to need changing at three in the morning is not whether the feature is on — it is which model it calls, which prompt it uses, and how much it is allowed to do without asking. Why the usual flag is not enough A conventional feature flag answers one question with a boolean, and it is the right shape because a conventional feature has one failure mode: it is broken. An AI feature has several, and they want different responses. The provider is degraded — you want a different model, not the feature off. A prompt change regressed quality — you want the previous prompt, which is not a code deploy. The feature is fine but a specific customer’s data is producing bad output — you want it off for them and on for everyone else. Spend is running above forecast — you want the cheap model or the degraded path, not an outage. A single boolean answers none of these, so the response to each becomes a deploy, and a deploy is the slowest tool available at the moment you most need speed. There is a second reason, specific to this dependency. The behaviour you are flagging can change without you deploying anything, because the model is somebody else’s and it can be updated underneath you. Flags are usually a mechanism for controlling your own changes; here they are also the mechanism for reacting to changes you did not make, which is why detecting a provider-side behaviour change and having a flag to respond with are two halves of one control. Four things to flag separately Axis Description Feature on/off The ordinary flag. Per-tenant and per-segment, because the common case is a problem confined to one customer's data rather than a global outage. Model selection Which model each call site uses, as configuration. This is what lets you switch providers during an incident, run a canary on a new model, or drop to a cheaper one under budget pressure — without shipping code. Promp