标签：#MachineLearning

Modern AI looks like magic from the outside. You type a sentence and a machine writes back something coherent, finishes your function, or turns a paragraph into Japanese. It's tempting to assume something exotic is happening in there. It isn't. The architecture behind almost every model you've heard of rests on a handful of plain engineering fixes, each one invented to get around a specific, annoying problem. No single genius moment, no secret sauce. Just people noticing their networks were broken and patching them. This is the story of three of those patches. If you can read a stack trace, you can follow all three. The wall everyone hit Around 2014, the recipe for a smarter neural network seemed obvious: make it deeper. More layers meant more capacity, which should have meant better results. Except past a certain point it stopped working. Deeper networks got worse , and not in the way you'd guess. The tell was the training error. A 56-layer network did worse on the very data it was being trained on than a 20-layer one. That rules out the usual suspect, overfitting, because the deep network couldn't even memorize the answers in front of it. The problem wasn't capacity. The network just couldn't be trained. Two things were going wrong. The error signal that teaches each layer (the gradient) has to travel backward through every layer to reach the early ones. Push a number through dozens of layers and it tends to either shrink to nothing or blow up, so the early layers got almost no usable feedback. And even when you wrestled the signal into shape, the optimization itself got harder the deeper you went. So depth, the thing that was supposed to make networks powerful, was the thing breaking them. Here's how three ideas knocked that wall down. Idea one: give the signal a shortcut The first fix is almost insultingly simple. Instead of forcing every layer to transform its input, you let the input skip ahead and get added back in later. Picture a block of layers that takes

Modern AI looks like magic from the outside. You type a sentence and a machine writes back something coherent, finishes your function, or turns a paragraph into Japanese. It's tempting to assume something exotic is happening in there. It isn't. The architecture behind almost every model you've heard of rests on a handful of plain engineering fixes, each one invented to get around a specific, annoying problem. No single genius moment, no secret sauce. Just people noticing their networks were broken and patching them. This is the story of three of those patches. If you can read a stack trace, you can follow all three. The wall everyone hit Around 2014, the recipe for a smarter neural network seemed obvious: make it deeper. More layers meant more capacity, which should have meant better results. Except past a certain point it stopped working. Deeper networks got worse , and not in the way you'd guess. The tell was the training error. A 56-layer network did worse on the very data it was being trained on than a 20-layer one. That rules out the usual suspect, overfitting, because the deep network couldn't even memorize the answers in front of it. The problem wasn't capacity. The network just couldn't be trained. Two things were going wrong. The error signal that teaches each layer (the gradient) has to travel backward through every layer to reach the early ones. Push a number through dozens of layers and it tends to either shrink to nothing or blow up, so the early layers got almost no usable feedback. And even when you wrestled the signal into shape, the optimization itself got harder the deeper you went. So depth, the thing that was supposed to make networks powerful, was the thing breaking them. Here's how three ideas knocked that wall down. Idea one: give the signal a shortcut The first fix is almost insultingly simple. Instead of forcing every layer to transform its input, you let the input skip ahead and get added back in later. Picture a block of layers that takes

La biblioteca di Babele di Borges è diventata un'ossessione o metafora potente per pensare l'intelligenza artificiale contemporanea. Il racconto del 1941 descrive una biblioteca infinita composta da gallerie esagonali, dove ogni libro contiene ogni possibile combinazione di 25 simboli ortografici. In essa risiede "la minuta storia del futuro, le autobiografie degli arcangeli, il catalogo fedele della Biblioteca, migliaia e migliaia di cataloghi falsi, la dimostrazione della fallacia di questi cataloghi, la dimostrazione della fallacia del catalogo vero" — eppure la stragrande maggioranza dei volumi è pura cacofonia senza senso. Questo scenario anticipa con precisione inquietante il problema fondamentale dei Large Language Models (LLM). Come ha notato Léon Bottou, il modello linguistico perfetto permette di navigare una collezione infinita di testi plausibili semplicemente digitando le prime parole, ma "nulla distingue il vero dal falso, l'utile dall'ingannevole, il giusto dallo sbagliato". La risposta di ChatGPT o di un altro modello generativo è, in un certo senso, un libro estratto a caso dalla Biblioteca di Babele: statisticamente plausibile, grammaticalmente corretta, ma non necessariamente ancorata a una verità esterna. Jonathan Basile, creatore del sito libraryofbabel.info , ha esplicitamente distinto la sua creazione dall'intelligenza artificiale: "Babele è tutta espressione nella sua forma più irrazionale e decontestualizzata; preferisco pensarla come unintelligenza artificiale".Eppure, paradossalmente, l'IA contemporanea ci ha portato più vicini che mai a realizzare la Biblioteca di Babele: non più un universo fisico di libri, ma un universo digitale di testi generati all'istante, dove la verità è circondata da infinite variazioni di falsità. La lezione di Borges è duplice. Da un lato, l'IA come strumento di navigazione: usare il Natural Language Processing per estrarre parole inglesi dal "gibberish" della Biblioteca, come dimostra la funzione "Anglishize"

Hello, I'm Maneshwar. I'm building git-lrc, a Micro AI code reviewer that runs on every commit. It is...

Most agents drag their entire past into every turn. A better default: keep a thin index of what was said hot, and fetch only the few turns you actually need — intact, on demand. Code: github.com/NirajPandey05/jit_context There is a quiet assumption baked into how most agents handle memory: that more context is safer than less. If the model might need something, put it in the window. The conversation grows, every prior turn rides along on every new request, and we trust the model to find the part that matters. That assumption breaks twice. It breaks on cost , because an agent loop re-sends its whole window on every step — a hundred stale turns aren't paid for once, they're paid for on turn 101, 102, and every step after. And it breaks on quality , because models don't read a long window evenly. Relevant facts buried in the middle get underweighted; irrelevant bulk competes for attention with the thing that actually answers the question. Past a point, a bigger context produces a worse answer, not just a costlier one. So the interesting question isn't "how do we fit more in?" It's "how do we keep the window small and dense without losing the one old turn that matters?" This post is the design we built around that question — for the specific case of long conversation history — plus the benchmark we used to keep ourselves honest. 01 · The mechanism: a hot index over a cold store The design borrows directly from how computers have always managed memory that doesn't fit: a small fast tier that's always present, a large slow tier that holds the bulk, and a rule for moving things between them. Virtual memory pages between RAM and disk. We page between the context window and an external store — for attention instead of address space. Concretely, there are two tiers. The cold store holds every turn at full fidelity, keyed by id — nothing is thrown away. The hot index holds one compact entry per turn: a short summary, a little metadata (entities, whether the turn recorded a dec

If you've been following the AI market lately, you already know the headline numbers: DeepSeek V4 costs about 3% of what GPT-4o charges per token. GLM-4 runs benchmarks competitive with GPT-4 at roughly one-twentieth the price. Qwen delivers multilingual performance that rivals Claude for a rounding error in your cloud bill. The spreadsheets look incredible. The problem is actually using these models. Signing up for each provider means navigating Chinese-language dashboards, topping up separate wallets, managing six different API key formats, and dealing with SDKs that don't follow any consistent convention. Most developers give up after the second integration. That friction is why, despite the economics being objectively absurd in 2026, most teams still default to a single Western provider and eat the cost. AIWave exists to kill that friction. One API key. One endpoint. Fifty-plus models across eight Chinese labs, all speaking standard OpenAI-compatible format. Zero code changes to switch between DeepSeek, GLM, Qwen, MiniMax, and everything else. This post covers how the platform works under the hood, what the request lifecycle looks like, and how to integrate it in any language that can speak HTTP. The Fragmentation Problem, Quantified Before getting into the solution, here's what the Chinese LLM landscape actually looks like as of June 2026: Provider Flagship Model API Format Auth Method SDK Language DeepSeek V4-Pro Custom (DS format) Bearer token + signature Python, JS Zhipu GLM-4.5 OpenAI-compatible-ish JWT with expiry Python, Java Alibaba Qwen-3-Max DashScope (Alibaba) AK/SK + HMAC Python, Java, Go MiniMax MiniMax-Text-01 Custom REST API Key + Group ID Python Moonshot Kimi-K2 OpenAI-compatible API Key Python, JS Baidu ERNIE 4.5 Qianfan (Baidu) OAuth 2.0 Client Cred Python ByteDance Doubao-Pro Ark (Volcengine) IAM AK/SK + SigV4 Python, Go 01.AI Yi-Lightning OpenAI-compatible API Key Python Eight providers, seven different authentication schemes, four distinct A

Supervised vs. Unsupervised Machine Learning: How to Choose the Right Approach Supervised learning trains a model on data that's already labeled with the correct answer, so it learns to predict outcomes for new, unseen examples. Unsupervised learning works on unlabeled data and finds patterns or groupings on its own, without being told what the "right answer" looks like. Use supervised learning when you have historical examples of the outcome you want to predict; use unsupervised learning when you're trying to discover structure in data you don't yet understand. That's the short version. Here's what it actually means in practice, and how to know which one your project needs. What is supervised learning? In supervised learning, every training example comes with a label — the "correct answer" the model is trying to learn to predict. Feed a model thousands of emails, each tagged "spam" or "not spam," and it learns the patterns that separate the two. Once trained, it can label emails it's never seen before. The defining trait: you already know the outcome for your training data. You're not asking the model to discover something new — you're asking it to learn a pattern well enough to apply it to fresh cases. Common supervised tasks: Classification — sorting things into categories (spam vs. not spam, fraudulent vs. legitimate transaction) Regression — predicting a number (home price, next month's revenue) What is unsupervised learning? Unsupervised learning gets raw, unlabeled data and is asked to find structure in it — without anyone telling it what to look for. There's no "correct answer" to check against during training. The defining trait: you don't know the outcome in advance — you're trying to find it. A retailer might feed customer purchase histories into an unsupervised model not because they have a label called "customer segment" already assigned, but because they want the model to discover natural groupings on its own. Common unsupervised tasks: Clustering — gr

If you've been following the AI market lately, you already know the headline numbers: DeepSeek V4 costs about 3% of what GPT-4o charges per token. GLM-4 runs benchmarks competitive with GPT-4 at roughly one-twentieth the price. Qwen delivers multilingual performance that rivals Claude for a rounding error in your cloud bill. The spreadsheets look incredible. The problem is actually using these models. Signing up for each provider means navigating Chinese-language dashboards, topping up separate wallets, managing six different API key formats, and dealing with SDKs that don't follow any consistent convention. Most developers give up after the second integration. That friction is why, despite the economics being objectively absurd in 2026, most teams still default to a single Western provider and eat the cost. AIWave exists to kill that friction. One API key. One endpoint. Fifty-plus models across eight Chinese labs, all speaking standard OpenAI-compatible format. Zero code changes to switch between DeepSeek, GLM, Qwen, MiniMax, and everything else. This post covers how the platform works under the hood, what the request lifecycle looks like, and how to integrate it in any language that can speak HTTP. The Fragmentation Problem, Quantified Before getting into the solution, here's what the Chinese LLM landscape actually looks like as of June 2026: Provider Flagship Model API Format Auth Method SDK Language DeepSeek V4-Pro Custom (DS format) Bearer token + signature Python, JS Zhipu GLM-4.5 OpenAI-compatible-ish JWT with expiry Python, Java Alibaba Qwen-3-Max DashScope (Alibaba) AK/SK + HMAC Python, Java, Go MiniMax MiniMax-Text-01 Custom REST API Key + Group ID Python Moonshot Kimi-K2 OpenAI-compatible API Key Python, JS Baidu ERNIE 4.5 Qianfan (Baidu) OAuth 2.0 Client Cred Python ByteDance Doubao-Pro Ark (Volcengine) IAM AK/SK + SigV4 Python, Go 01.AI Yi-Lightning OpenAI-compatible API Key Python Eight providers, seven different authentication schemes, four distinct A

Stop Fighting Scikit-Learn Pipelines: How Metadata Routing Fixes Sample Weights & Groups A couple of months ago, I stumbled upon this video by Vincent D. Warmerdam about metadata routing in scikit-learn. I'll be honest, I had no idea what "metadata routing" even meant, but Vincent's explanation completely changed how I think about building ML pipelines. The video showed me that one of the most frustrating problems in scikit-learn; passing sample weights and groups through complex pipelines finally had an elegant solution. It piqued my curiosity enough that I dove deep into the feature, tested it extensively, and honestly, I was surprised by how little coverage this gets in technical blogs and articles. So I figured, why not write about it myself and share what I learned? If you've ever struggled with imbalanced datasets, grouped cross-validation, or just wanted to pass custom information through your pipelines, this article is for you. Let's start from the very beginning. What is "Metadata" in Machine Learning? Let's start with a concrete example. You're building a credit card fraud detection model with this data: # Your training data X = transaction_features # Amount, merchant, time, location, etc. y = is_fraud # 0 = legitimate, 1 = fraud # But you also have additional information: sample_weights = [ 1.0 , 1.0 , 10.0 , 1.0 , ...] # Fraud transactions weighted 10x customer_ids = [ 101 , 102 , 101 , 103 , ...] # Which customer made each transaction Metadata is the "extra information" beyond your features (X) and labels (y): sample_weight : How important is each transaction? (Fraud = 10x more important) groups : Which customer does each transaction belong to? (For proper cross-validation) Custom metadata : Transaction timestamps, confidence scores, data quality flags, etc. Why Metadata Matters: The Credit Card Fraud Problem Imagine you're building a fraud detection system for a financial company. You have: Imbalanced data : 99% legitimate transactions, 1% fraudulent T

"Intelligence isn't about searching everywhere—it's about knowing where not to search." Artificial Intelligence is often associated with neural networks, large language models, and autonomous systems. But long before modern generative AI, computer scientists were solving a much deeper question: How do intelligent systems make decisions efficiently? Whether you're building search algorithms, recommendation systems, autonomous robots, or distributed systems, the architecture of intelligence teaches timeless lessons about solving problems under uncertainty. Let's explore five powerful ideas that shaped AI—and why they matter far beyond computer science. ✈️ 1. The Pilot's Dilemma: Why Blind Search Fails Imagine you're a pilot. Suddenly, one of your engines fails. In the next few seconds, there are hundreds of switches, buttons, and controls available. If you treated every control equally, you'd spend precious time trying random combinations. That is exactly how uninformed search works. Algorithms like: Breadth-First Search (BFS) Depth-First Search (DFS) have no knowledge of where the solution might be. They simply explore. Start ├── Option A ├── Option B ├── Option C └── ... The larger the search space becomes, the less practical this strategy is. A pilot doesn't blindly flip switches. They use additional knowledge : Engine pressure Fuel flow Hydraulic readings Warning systems Those clues dramatically reduce the number of possibilities. This is exactly what AI calls Informed Search . Instead of exploring everything, intelligent systems use knowledge to eliminate impossible paths before searching them. 🧠 2. Heuristics: The Cheat Code of Intelligence The secret behind informed search is something called a heuristic . A heuristic is simply an educated estimate. Mathematically, h(n) represents the estimated cost from the current state to the goal. One important rule always holds: h(goal) = 0 Once we've reached the goal, there's no remaining cost. Example: Finding Bucharest

Your retriever returned the right documents. The similarity scores look fine. The answer is still wrong. If you've shipped RAG, you've seen this — and it's the failure that survives every retrieval upgrade. What everyone tries Reranker. Higher top-k. Hybrid search. A better embedding model. All of these chase the same goal: documents more similar to the query. They help when the right document wasn't being retrieved. They do nothing when the right document was retrieved and the answer is still wrong. Why it doesn't work Similarity answers "is this chunk about the same topic?" It does not answer "does this chunk contain the facts needed to support the answer?" Those come apart constantly. A chunk can be highly similar — same vocabulary, same subject — and contain nothing that actually grounds the answer. Hand the model a pile of on-topic text and it will produce a fluent, plausible, even cited-looking answer. The grounding is cosmetic: the text was nearby, not load-bearing. High similarity with a wrong answer isn't a contradiction. You asked retrieval to find related text. It did. Nobody asked whether the text was enough. The one shift Stop treating retrieval output as evidence. Treat it as candidate material that has to pass an explicit evidence check before it can support an answer. Put a step between retrieval and generation: does the retrieved set actually contain the facts this answer requires? If not, abstain. When the documents don't contain the facts, the system should return nothing rather than a confident guess. Relevant context in, only sufficient evidence allowed through. That's the line between a RAG demo and a RAG system you can trust in production. I write about the three boundaries where production RAG dies — query, evidence, output — from the angle of shipping under security and model constraints. Read the full version on my blog , where this connects to the practical RAG Failure Diagnosis Kit for teams debugging production RAG.

PSYONIC's prosthetic touch data is now training ABB robots. Gatik signed the first Fortune 50 commercial autonomous freight contract with PepsiCo. Burro drove Physical AI onto the construction site. Experts set $20k as the humanoid price target. And someone just called Edge AI the Windows of robotics. This week, Physical AI crossed three invisible lines at once. A company that makes prosthetic hands figured out that the touch data from amputees is exactly what industrial robots need to learn how to grip. A Fortune 50 company signed not a pilot but a commercial contract for autonomous freight. A 44-horsepower robot drove off the warehouse floor and onto the construction site. And two separate conversations about software and pricing suggest that the next wave of robotics adoption will be driven by access, not capability. Here is what happened, and why it matters beyond the headlines. Value Description Fortune 50 PepsiCo becomes first to sign a commercial contract for autonomous freight with Gatik $20k Target price point for humanoid robots, Robotics Summit consensus: achievable by 2028–2030 1M hours Burro's field experience backing the Grande 44 autonomous outdoor platform 100+ Pressure sensors per fingertip in PSYONIC's Ability Hand, now training ABB GoFa A Prosthetic Hand Is Now Teaching an Industrial Robot How to Grip The standard approach to teaching a robot how to handle objects has been simulation, teleoperation, or labor-intensive physical demonstrations. PSYONIC and ABB just introduced a different source of data : the hands of people who have already learned to feel again. PSYONIC's Ability Hand is a prosthetic with more than 100 pressure sensors per fingertip . The company has been collecting kinesthetic data from users with upper-limb amputations. That data, which captures how a human hand adjusts grip pressure, contact area, and force across thousands of everyday tasks, is now being fed as training data into ABB GoFa robot arm models. The implication is no

How I Cut My Multimodal AI Costs by 97% — A Freelancer's Guide Last month I almost killed a side gig because of a single line item on an invoice. A client wanted me to build a document-processing tool that could read scanned PDFs, pull text out of photos, and answer questions about charts. Easy enough — except I'd quoted the job assuming I'd use GPT-4o for the vision work. When I actually ran the numbers, I realized the API bill would eat my entire margin. I'd be working for free. Maybe worse. So I did what every freelancer does when the big-name vendor gets too expensive: I went hunting. And I landed on Global API, which routes to a bunch of multimodal models I've honestly never heard clients talk about. After a few weeks of testing, I figured out which ones are worth my billable hours and which ones aren't. This is everything I learned, plus the exact code I'm shipping to clients. Why Multimodal Even Matters for Solo Devs Two years ago, "multimodal" was a buzzword you'd hear at conferences. In 2026 it's table stakes. I've personally used vision models to: OCR receipts for an expense-tracking app (boring but pays the rent) Convert screenshots of legacy code into editable source for a Y2K-era company migration Read bar charts from PDF reports for a finance client who hates spreadsheets Analyze medical imaging samples for a startup MVP (this one was scary) Every one of those jobs started as a quick conversation with a prospect and turned into real invoices because I could say yes. The bottleneck was never capability — it was always cost. When GPT-4o charges north of $10/M output tokens, a single 2,000-token response on a tricky chart costs me about two cents. Multiply by 10,000 images per month and you've got a $200 API line item before you've paid yourself. That's a problem when the whole job is worth $400. So I tested every multimodal model I could find on Global API. Here's the lineup I ended up evaluating. The Contenders Nine models, three providers, one freelanc

Build-in-public. This is the real architecture behind running ~50 local AI agents on 6GB of VRAM — one GPU lock, an eviction watchdog, a resource governor, and a model router. Originally posted on my blog. The question I get most often is some version of "there's no way you run that many agents on a 6GB laptop GPU." The honest answer: not the way you're picturing it. I don't run 50 models at once. I run one model at a time, very deliberately — and most of the engineering is about scheduling, not inference. Here's the actual architecture. The hard constraint: 6GB of VRAM A single consumer GPU with 6GB of VRAM holds roughly one 7B-parameter model at a usable quantization. Two at once? It thrashes — the GPU starts swapping, latency explodes, and eventually a driver out-of-memory can take the whole machine down. I've had the desktop freeze from exactly that. So the first design rule wrote itself: only one heavy model is allowed on the GPU at any moment. That sounds limiting. It isn't — because almost nothing I run is latency-sensitive. A blog post that publishes at 7am doesn't care if it was generated at 6:52 or 6:58. Once you accept that your AI workforce is a batch system, not a chat window, the whole problem changes shape. A lock, not a crowd Every agent that needs the GPU has to take a lock first. It's a simple file-based queue with: FIFO ordering PID-based ownership Stale-lock detection, so a crashed job can't wedge the line forever If an agent can't get the lock within its timeout, it skips gracefully and tries again on its next scheduled run instead of piling up. So at 50 agents, what's really happening is: dozens of cron-scheduled Python workers wake up throughout the day, and the ones that need the model form an orderly line for it. The fleet is huge; the GPU contention is always exactly one. That's the trick. It's less "50 models" and more "50 employees sharing one very busy workstation, politely." Eviction and a VRAM watchdog Even with the lock, idle models l

I recently calibrated a recovery-rate model that had only two weak features. Its point accuracy was almost nothing — R² basically zero. I expected its uncertainty estimates to be junk too. They weren't: the 90% conformal prediction intervals covered ~89% of held-out outcomes. Valid, just wide . That surprised me enough to nail it down, because it contradicts a belief a lot of us carry around: "my model isn't accurate, so I can't trust its uncertainty." For split conformal prediction, that's backwards. Here's the precise statement, a runnable demo, and the one caveat that actually bites. Coverage is a property of the procedure, not the model Split conformal prediction gives a distribution-free, finite-sample marginal coverage guarantee : P( Y ∈ Ĉ(X) ) ≥ 1 − α and it holds for any point model, as long as the calibration and test data are exchangeable. The model is a black box. You fit it however you like, then on a held-out calibration set you take the (1−α) quantile of the absolute residuals, and that quantile becomes the half-width of your intervals. Nowhere does that construction require the model to be good. A bad model just has large residuals, so the calibration quantile is large, so the intervals are wide — wide enough to still cover at the stated rate. Accuracy doesn't buy you validity ; it buys you efficiency (narrower intervals at the same coverage). The demo (numbers are reproducible, seed fixed) Same dataset and target, three models from strong to useless, target coverage 90%: model R² marginal coverage mean interval width gradient boosting 0.741 0.895 5.39 weak linear (1 noisy feature) 0.061 0.905 10.39 predict-the-mean −0.000 0.907 10.83 All three land at ~90% coverage. The only thing that changes is width: the good model's intervals are half as wide . That's the whole story in one table — validity is constant, efficiency tracks accuracy. import numpy as np from sklearn.linear_model import LinearRegression from sklearn.ensemble import GradientBoostingReg

In the previous series, when we optimized our neural network, we had to write quite a bit of training code ourselves. First, we created an optimizer object that used Stochastic Gradient Descent (SGD) to optimize final_bias . Then we wrote loops to calculate the derivatives required for gradient descent. We trained the model for up to 100 epochs . For each training example, we: Ran the input through the neural network to get a prediction. Calculated the loss. Calculated the derivatives of the loss function. After processing all three training points, we used: optimizer . step () to take a small step toward a better value for final_bias . Then we used: optimizer . zero_grad () to clear the accumulated gradients before starting the next epoch. All of this required a considerable amount of training code. Let's see how Lightning helps simplify this process. Organizing Training Logic with Lightning Previously, we created a class to store the weights, biases, and the forward() function. The optimization-related code was written separately outside the class. With Lightning, we can keep all of this logic in one place. We start by creating the class as usual, and then add a few new methods. Configuring the Optimizer The first method is configure_optimizers() . def configure_optimizers ( self ): return SGD ( self . parameters (), lr = self . learning_rate ) This method tells Lightning how the neural network should be optimized. The learning rate is stored in the self.learning_rate variable that we defined earlier. Defining a Training Step Next, we add a method called training_step() . def training_step ( self , batch , batch_idx ): input_i , label_i = batch output_i = self . forward ( input_i ) loss = ( output_i - label_i ) ** 2 return loss This method receives: A batch of training data from the DataLoader. The index of that batch. Inside the method, we: Extract the input and label from the batch. Run the input through the neural network. Calculate the loss using the squared r

"How does ChatGPT think ?" It doesn't. The entire mechanism behind every chatbot is almost anticlimactic: it predicts one next word , adds it, and repeats. I built a tiny interactive predictor so you can be the model — and it explains both the magic and the flaws. 🔮 Be the model: https://dev48v.infy.uk/ai/days/day6-next-token.html This is Day 6 of AIFromZero — AI literacy, one concept a day, no code to follow. 1. It only predicts the NEXT word Given everything so far, the model outputs a probability for every possible next word, picks one, appends it, and runs again with the longer text. Paragraphs, code, poems — all of it is this one step on repeat. "the cat sat on the ___" → P(mat) high, P(bird) low 2. It's a probability over the WHOLE vocabulary The output isn't one word — it's a number for every word it knows (100,000+ for a real model). Most are near zero; a handful are plausible. The bars in the demo are that distribution, over a tiny vocabulary. 3. Autoregression: feed the output back in After picking a word, it becomes part of the input for the next prediction. Predict → append → predict again. Because each new word conditions on all the previous ones, short local choices add up to coherent long text. 4. Temperature = the creativity dial Once you have probabilities, how do you choose? Temperature reshapes them before sampling: Near 0: the top word always wins — safe, repetitive. High: the odds flatten, so rarer words get a real chance — creative, error-prone. p = p ** ( 1 / temperature ); // then renormalise and sample Drag the slider in the demo and watch the bars sharpen or even out. That one knob is what an API calls "creativity." 5. Where do the probabilities come from? In my toy, from counting which word followed which in a few sentences (a "bigram" with 1-word memory). A real LLM replaces the counting with a giant neural network trained on much of the internet, and its memory spans thousands of words. The mechanism is identical — only the quality of th

Pick the wrong loss function and your model optimises the wrong thing — perfectly. The loss is the single number training tries to shrink, so it quietly defines what "wrong" even means. I built an interactive visualiser of MSE, MAE, and cross-entropy so you can see why the choice matters. 🎯 Drag the prediction: https://dev48v.infy.uk/dl/day6-loss-functions.html This is Day 6 of DeepLearningFromZero. Loss = one number for "how wrong" The network's output is compared to the truth and collapsed into one scalar. Everything in training exists to make that number smaller. Choose the loss and you've defined the network's entire goal. MSE — square the error (regression) const mse = ( pred , y ) => ( pred - y ) ** 2 ; Squaring means off-by-4 hurts 16×, off-by-1 hurts 1×. MSE obsesses over large errors — great when big misses are unacceptable, risky when outliers will drag the model around. MAE — absolute error, outlier-robust const mae = ( pred , y ) => Math . abs ( pred - y ); Linear penalty: off-by-4 hurts exactly 4× off-by-1. One wild outlier can't dominate. The trade-off is a constant gradient, so it can be slower and less precise near the answer. Cross-entropy — for classification When the output is a probability, you don't use MSE. Cross-entropy rewards confident-and-right and brutally punishes confident-and-wrong: const bce = ( p , y ) => - ( y * Math . log ( p ) + ( 1 - y ) * Math . log ( 1 - p )); Predict 1% for the true class and the loss screams toward infinity. In the demo, switch to Classification and slide p toward 0 to watch it explode. The slope is what learning actually uses Backprop doesn't follow the loss value — it follows the loss's gradient (slope) downhill. That's why the shape matters: cross-entropy's steep slope when very wrong gives a strong corrective push, helping classifiers learn faster than MSE would. grad = dLoss / dPred ; // gradient descent steps along this Choosing the loss is a design decision Predicting a price? MSE or MAE. Yes/no? Binary

Naive Bayes ran real spam filters for years, and it's the rare ML model whose "training" is just counting . No gradient descent, no iterations — count words, apply Bayes' rule, multiply. I built one from scratch and visualised exactly which words push a message toward spam. 📨 Interactive demo (type a message): https://dev48v.infy.uk/ml/day6-naive-bayes.html This is Day 6 of MachineLearningFromZero — algorithms from scratch, no scikit-learn. 1. Bag of words — order doesn't matter Naive Bayes treats a message as a set of words. "free cash now" and "now cash free" look identical to it. That throws away grammar, but for spam detection the words present matter far more than their order — and it makes the math tiny. 2. Training = counting For every word, how often does it appear in spam vs ham? for ( const { text , label } of trainingData ) for ( const w of tokenize ( text )) counts [ label ][ w ] = ( counts [ label ][ w ] || 0 ) + 1 ; free and click flood spam; meeting and tomorrow live in ham. One pass over the data, done. 3. Bayes' rule flips the question You measured P(words | spam) , but you want P(spam | words) . Bayes flips it: P(spam | words) ∝ P(spam) × P(words | spam) P(spam) is the prior (how common spam is); the likelihood multiplies in the word evidence. 4. "Naive" = pretend words are independent The trick that makes it fast: assume each word is independent given the class, so the likelihood is just a product: P(words | spam) = P(w1|spam) × P(w2|spam) × ... Real words aren't independent ("credit" and "card" co-occur), so it's a naive lie — but the classification still lands right astonishingly often. 5. Smoothing + logs keep it stable Two practical fixes. Add 1 to every count (Laplace smoothing) so an unseen word doesn't zero out the whole product. And add logarithms instead of multiplying tiny probabilities, which would underflow to 0: score [ label ] = Math . log ( prior [ label ]); for ( const w of words ) score [ label ] += Math . log (( counts [ label ][

vibe citing: how KPMG used AI to write a report about AI and AI made them look like fools by t474-r0b07 There are companies that charge you to tell you how to use AI responsibly. KPMG is one of them. 250,000 employees. 138 countries. Decades advising governments and corporations on how to avoid costly mistakes. In October 2025 they published a report titled "Total Experience: Redefining Excellence in the Age of Agentic AI" . They wrote it with AI. The AI invented 88% of the sources. Nobody verified anything. They published it anyway. // what is "agentic AI" — because the title matters An agentic AI is not a chatbot. Not the assistant that answers your questions. It's a system that makes decisions and executes actions on its own, without a human approving each step. You give it an objective and it acts, corrects, moves forward. It's the product everyone in the tech sector was selling in 2025. KPMG was selling it too. That's why they needed a report proving their clients were already using it. Spoiler: they weren't. And the report invented it anyway. // the forensic analysis GPTZero — a company specialized in detecting AI-generated content — ran a full audit on the report. First: what is an AI hallucination, because the term is going to come up a lot. When a language model doesn't have the information you ask for, it doesn't say "I don't know." It generates a response that sounds correct. It invents with the same confidence it would use if it actually knew the truth. Perfect format. False content. No warning. That's a hallucination. Now the numbers from the KPMG report: TOTAL CITATIONS: 45 REAL CITATIONS: 5 INVENTED CITATIONS: 40 ACCURACY RATE: 11.1% 40 of 45 citations have invented titles, authors that don't exist, or sources that don't say what KPMG claimed they said. Half of the factual claims in the report are false or misattributed. A firm that charges for intellectual rigor published a document with 11% accuracy. // the organizations that read the report and sai