标签：#Mac

La biblioteca di Babele di Borges è diventata un'ossessione o metafora potente per pensare l'intelligenza artificiale contemporanea. Il racconto del 1941 descrive una biblioteca infinita composta da gallerie esagonali, dove ogni libro contiene ogni possibile combinazione di 25 simboli ortografici. In essa risiede "la minuta storia del futuro, le autobiografie degli arcangeli, il catalogo fedele della Biblioteca, migliaia e migliaia di cataloghi falsi, la dimostrazione della fallacia di questi cataloghi, la dimostrazione della fallacia del catalogo vero" — eppure la stragrande maggioranza dei volumi è pura cacofonia senza senso. Questo scenario anticipa con precisione inquietante il problema fondamentale dei Large Language Models (LLM). Come ha notato Léon Bottou, il modello linguistico perfetto permette di navigare una collezione infinita di testi plausibili semplicemente digitando le prime parole, ma "nulla distingue il vero dal falso, l'utile dall'ingannevole, il giusto dallo sbagliato". La risposta di ChatGPT o di un altro modello generativo è, in un certo senso, un libro estratto a caso dalla Biblioteca di Babele: statisticamente plausibile, grammaticalmente corretta, ma non necessariamente ancorata a una verità esterna. Jonathan Basile, creatore del sito libraryofbabel.info , ha esplicitamente distinto la sua creazione dall'intelligenza artificiale: "Babele è tutta espressione nella sua forma più irrazionale e decontestualizzata; preferisco pensarla come unintelligenza artificiale".Eppure, paradossalmente, l'IA contemporanea ci ha portato più vicini che mai a realizzare la Biblioteca di Babele: non più un universo fisico di libri, ma un universo digitale di testi generati all'istante, dove la verità è circondata da infinite variazioni di falsità. La lezione di Borges è duplice. Da un lato, l'IA come strumento di navigazione: usare il Natural Language Processing per estrarre parole inglesi dal "gibberish" della Biblioteca, come dimostra la funzione "Anglishize"

At WWDC 26, Apple announced the Core AI framework, the official successor to Core ML. It is designed to allow developers to run large language models and generative AI entirely on-device, supporting both custom-converted PyTorch models and pre-optimized open-source models. By Sergio De Simone

Hello, I'm Maneshwar. I'm building git-lrc, a Micro AI code reviewer that runs on every commit. It is...

Most agents drag their entire past into every turn. A better default: keep a thin index of what was said hot, and fetch only the few turns you actually need — intact, on demand. Code: github.com/NirajPandey05/jit_context There is a quiet assumption baked into how most agents handle memory: that more context is safer than less. If the model might need something, put it in the window. The conversation grows, every prior turn rides along on every new request, and we trust the model to find the part that matters. That assumption breaks twice. It breaks on cost , because an agent loop re-sends its whole window on every step — a hundred stale turns aren't paid for once, they're paid for on turn 101, 102, and every step after. And it breaks on quality , because models don't read a long window evenly. Relevant facts buried in the middle get underweighted; irrelevant bulk competes for attention with the thing that actually answers the question. Past a point, a bigger context produces a worse answer, not just a costlier one. So the interesting question isn't "how do we fit more in?" It's "how do we keep the window small and dense without losing the one old turn that matters?" This post is the design we built around that question — for the specific case of long conversation history — plus the benchmark we used to keep ourselves honest. 01 · The mechanism: a hot index over a cold store The design borrows directly from how computers have always managed memory that doesn't fit: a small fast tier that's always present, a large slow tier that holds the bulk, and a rule for moving things between them. Virtual memory pages between RAM and disk. We page between the context window and an external store — for attention instead of address space. Concretely, there are two tiers. The cold store holds every turn at full fidelity, keyed by id — nothing is thrown away. The hot index holds one compact entry per turn: a short summary, a little metadata (entities, whether the turn recorded a dec

If you've been following the AI market lately, you already know the headline numbers: DeepSeek V4 costs about 3% of what GPT-4o charges per token. GLM-4 runs benchmarks competitive with GPT-4 at roughly one-twentieth the price. Qwen delivers multilingual performance that rivals Claude for a rounding error in your cloud bill. The spreadsheets look incredible. The problem is actually using these models. Signing up for each provider means navigating Chinese-language dashboards, topping up separate wallets, managing six different API key formats, and dealing with SDKs that don't follow any consistent convention. Most developers give up after the second integration. That friction is why, despite the economics being objectively absurd in 2026, most teams still default to a single Western provider and eat the cost. AIWave exists to kill that friction. One API key. One endpoint. Fifty-plus models across eight Chinese labs, all speaking standard OpenAI-compatible format. Zero code changes to switch between DeepSeek, GLM, Qwen, MiniMax, and everything else. This post covers how the platform works under the hood, what the request lifecycle looks like, and how to integrate it in any language that can speak HTTP. The Fragmentation Problem, Quantified Before getting into the solution, here's what the Chinese LLM landscape actually looks like as of June 2026: Provider Flagship Model API Format Auth Method SDK Language DeepSeek V4-Pro Custom (DS format) Bearer token + signature Python, JS Zhipu GLM-4.5 OpenAI-compatible-ish JWT with expiry Python, Java Alibaba Qwen-3-Max DashScope (Alibaba) AK/SK + HMAC Python, Java, Go MiniMax MiniMax-Text-01 Custom REST API Key + Group ID Python Moonshot Kimi-K2 OpenAI-compatible API Key Python, JS Baidu ERNIE 4.5 Qianfan (Baidu) OAuth 2.0 Client Cred Python ByteDance Doubao-Pro Ark (Volcengine) IAM AK/SK + SigV4 Python, Go 01.AI Yi-Lightning OpenAI-compatible API Key Python Eight providers, seven different authentication schemes, four distinct A

Supervised vs. Unsupervised Machine Learning: How to Choose the Right Approach Supervised learning trains a model on data that's already labeled with the correct answer, so it learns to predict outcomes for new, unseen examples. Unsupervised learning works on unlabeled data and finds patterns or groupings on its own, without being told what the "right answer" looks like. Use supervised learning when you have historical examples of the outcome you want to predict; use unsupervised learning when you're trying to discover structure in data you don't yet understand. That's the short version. Here's what it actually means in practice, and how to know which one your project needs. What is supervised learning? In supervised learning, every training example comes with a label — the "correct answer" the model is trying to learn to predict. Feed a model thousands of emails, each tagged "spam" or "not spam," and it learns the patterns that separate the two. Once trained, it can label emails it's never seen before. The defining trait: you already know the outcome for your training data. You're not asking the model to discover something new — you're asking it to learn a pattern well enough to apply it to fresh cases. Common supervised tasks: Classification — sorting things into categories (spam vs. not spam, fraudulent vs. legitimate transaction) Regression — predicting a number (home price, next month's revenue) What is unsupervised learning? Unsupervised learning gets raw, unlabeled data and is asked to find structure in it — without anyone telling it what to look for. There's no "correct answer" to check against during training. The defining trait: you don't know the outcome in advance — you're trying to find it. A retailer might feed customer purchase histories into an unsupervised model not because they have a label called "customer segment" already assigned, but because they want the model to discover natural groupings on its own. Common unsupervised tasks: Clustering — gr

If you've been following the AI market lately, you already know the headline numbers: DeepSeek V4 costs about 3% of what GPT-4o charges per token. GLM-4 runs benchmarks competitive with GPT-4 at roughly one-twentieth the price. Qwen delivers multilingual performance that rivals Claude for a rounding error in your cloud bill. The spreadsheets look incredible. The problem is actually using these models. Signing up for each provider means navigating Chinese-language dashboards, topping up separate wallets, managing six different API key formats, and dealing with SDKs that don't follow any consistent convention. Most developers give up after the second integration. That friction is why, despite the economics being objectively absurd in 2026, most teams still default to a single Western provider and eat the cost. AIWave exists to kill that friction. One API key. One endpoint. Fifty-plus models across eight Chinese labs, all speaking standard OpenAI-compatible format. Zero code changes to switch between DeepSeek, GLM, Qwen, MiniMax, and everything else. This post covers how the platform works under the hood, what the request lifecycle looks like, and how to integrate it in any language that can speak HTTP. The Fragmentation Problem, Quantified Before getting into the solution, here's what the Chinese LLM landscape actually looks like as of June 2026: Provider Flagship Model API Format Auth Method SDK Language DeepSeek V4-Pro Custom (DS format) Bearer token + signature Python, JS Zhipu GLM-4.5 OpenAI-compatible-ish JWT with expiry Python, Java Alibaba Qwen-3-Max DashScope (Alibaba) AK/SK + HMAC Python, Java, Go MiniMax MiniMax-Text-01 Custom REST API Key + Group ID Python Moonshot Kimi-K2 OpenAI-compatible API Key Python, JS Baidu ERNIE 4.5 Qianfan (Baidu) OAuth 2.0 Client Cred Python ByteDance Doubao-Pro Ark (Volcengine) IAM AK/SK + SigV4 Python, Go 01.AI Yi-Lightning OpenAI-compatible API Key Python Eight providers, seven different authentication schemes, four distinct A

Stop Fighting Scikit-Learn Pipelines: How Metadata Routing Fixes Sample Weights & Groups A couple of months ago, I stumbled upon this video by Vincent D. Warmerdam about metadata routing in scikit-learn. I'll be honest, I had no idea what "metadata routing" even meant, but Vincent's explanation completely changed how I think about building ML pipelines. The video showed me that one of the most frustrating problems in scikit-learn; passing sample weights and groups through complex pipelines finally had an elegant solution. It piqued my curiosity enough that I dove deep into the feature, tested it extensively, and honestly, I was surprised by how little coverage this gets in technical blogs and articles. So I figured, why not write about it myself and share what I learned? If you've ever struggled with imbalanced datasets, grouped cross-validation, or just wanted to pass custom information through your pipelines, this article is for you. Let's start from the very beginning. What is "Metadata" in Machine Learning? Let's start with a concrete example. You're building a credit card fraud detection model with this data: # Your training data X = transaction_features # Amount, merchant, time, location, etc. y = is_fraud # 0 = legitimate, 1 = fraud # But you also have additional information: sample_weights = [ 1.0 , 1.0 , 10.0 , 1.0 , ...] # Fraud transactions weighted 10x customer_ids = [ 101 , 102 , 101 , 103 , ...] # Which customer made each transaction Metadata is the "extra information" beyond your features (X) and labels (y): sample_weight : How important is each transaction? (Fraud = 10x more important) groups : Which customer does each transaction belong to? (For proper cross-validation) Custom metadata : Transaction timestamps, confidence scores, data quality flags, etc. Why Metadata Matters: The Credit Card Fraud Problem Imagine you're building a fraud detection system for a financial company. You have: Imbalanced data : 99% legitimate transactions, 1% fraudulent T

"Intelligence isn't about searching everywhere—it's about knowing where not to search." Artificial Intelligence is often associated with neural networks, large language models, and autonomous systems. But long before modern generative AI, computer scientists were solving a much deeper question: How do intelligent systems make decisions efficiently? Whether you're building search algorithms, recommendation systems, autonomous robots, or distributed systems, the architecture of intelligence teaches timeless lessons about solving problems under uncertainty. Let's explore five powerful ideas that shaped AI—and why they matter far beyond computer science. ✈️ 1. The Pilot's Dilemma: Why Blind Search Fails Imagine you're a pilot. Suddenly, one of your engines fails. In the next few seconds, there are hundreds of switches, buttons, and controls available. If you treated every control equally, you'd spend precious time trying random combinations. That is exactly how uninformed search works. Algorithms like: Breadth-First Search (BFS) Depth-First Search (DFS) have no knowledge of where the solution might be. They simply explore. Start ├── Option A ├── Option B ├── Option C └── ... The larger the search space becomes, the less practical this strategy is. A pilot doesn't blindly flip switches. They use additional knowledge : Engine pressure Fuel flow Hydraulic readings Warning systems Those clues dramatically reduce the number of possibilities. This is exactly what AI calls Informed Search . Instead of exploring everything, intelligent systems use knowledge to eliminate impossible paths before searching them. 🧠 2. Heuristics: The Cheat Code of Intelligence The secret behind informed search is something called a heuristic . A heuristic is simply an educated estimate. Mathematically, h(n) represents the estimated cost from the current state to the goal. One important rule always holds: h(goal) = 0 Once we've reached the goal, there's no remaining cost. Example: Finding Bucharest

Your retriever returned the right documents. The similarity scores look fine. The answer is still wrong. If you've shipped RAG, you've seen this — and it's the failure that survives every retrieval upgrade. What everyone tries Reranker. Higher top-k. Hybrid search. A better embedding model. All of these chase the same goal: documents more similar to the query. They help when the right document wasn't being retrieved. They do nothing when the right document was retrieved and the answer is still wrong. Why it doesn't work Similarity answers "is this chunk about the same topic?" It does not answer "does this chunk contain the facts needed to support the answer?" Those come apart constantly. A chunk can be highly similar — same vocabulary, same subject — and contain nothing that actually grounds the answer. Hand the model a pile of on-topic text and it will produce a fluent, plausible, even cited-looking answer. The grounding is cosmetic: the text was nearby, not load-bearing. High similarity with a wrong answer isn't a contradiction. You asked retrieval to find related text. It did. Nobody asked whether the text was enough. The one shift Stop treating retrieval output as evidence. Treat it as candidate material that has to pass an explicit evidence check before it can support an answer. Put a step between retrieval and generation: does the retrieved set actually contain the facts this answer requires? If not, abstain. When the documents don't contain the facts, the system should return nothing rather than a confident guess. Relevant context in, only sufficient evidence allowed through. That's the line between a RAG demo and a RAG system you can trust in production. I write about the three boundaries where production RAG dies — query, evidence, output — from the angle of shipping under security and model constraints. Read the full version on my blog , where this connects to the practical RAG Failure Diagnosis Kit for teams debugging production RAG.

Apple’s fall macOS release will let you build Shortcuts by typing what you want to happen. But Claude Code and Codex users don’t have to wait.

PSYONIC's prosthetic touch data is now training ABB robots. Gatik signed the first Fortune 50 commercial autonomous freight contract with PepsiCo. Burro drove Physical AI onto the construction site. Experts set $20k as the humanoid price target. And someone just called Edge AI the Windows of robotics. This week, Physical AI crossed three invisible lines at once. A company that makes prosthetic hands figured out that the touch data from amputees is exactly what industrial robots need to learn how to grip. A Fortune 50 company signed not a pilot but a commercial contract for autonomous freight. A 44-horsepower robot drove off the warehouse floor and onto the construction site. And two separate conversations about software and pricing suggest that the next wave of robotics adoption will be driven by access, not capability. Here is what happened, and why it matters beyond the headlines. Value Description Fortune 50 PepsiCo becomes first to sign a commercial contract for autonomous freight with Gatik $20k Target price point for humanoid robots, Robotics Summit consensus: achievable by 2028–2030 1M hours Burro's field experience backing the Grande 44 autonomous outdoor platform 100+ Pressure sensors per fingertip in PSYONIC's Ability Hand, now training ABB GoFa A Prosthetic Hand Is Now Teaching an Industrial Robot How to Grip The standard approach to teaching a robot how to handle objects has been simulation, teleoperation, or labor-intensive physical demonstrations. PSYONIC and ABB just introduced a different source of data : the hands of people who have already learned to feel again. PSYONIC's Ability Hand is a prosthetic with more than 100 pressure sensors per fingertip . The company has been collecting kinesthetic data from users with upper-limb amputations. That data, which captures how a human hand adjusts grip pressure, contact area, and force across thousands of everyday tasks, is now being fed as training data into ABB GoFa robot arm models. The implication is no

How I Cut My Multimodal AI Costs by 97% — A Freelancer's Guide Last month I almost killed a side gig because of a single line item on an invoice. A client wanted me to build a document-processing tool that could read scanned PDFs, pull text out of photos, and answer questions about charts. Easy enough — except I'd quoted the job assuming I'd use GPT-4o for the vision work. When I actually ran the numbers, I realized the API bill would eat my entire margin. I'd be working for free. Maybe worse. So I did what every freelancer does when the big-name vendor gets too expensive: I went hunting. And I landed on Global API, which routes to a bunch of multimodal models I've honestly never heard clients talk about. After a few weeks of testing, I figured out which ones are worth my billable hours and which ones aren't. This is everything I learned, plus the exact code I'm shipping to clients. Why Multimodal Even Matters for Solo Devs Two years ago, "multimodal" was a buzzword you'd hear at conferences. In 2026 it's table stakes. I've personally used vision models to: OCR receipts for an expense-tracking app (boring but pays the rent) Convert screenshots of legacy code into editable source for a Y2K-era company migration Read bar charts from PDF reports for a finance client who hates spreadsheets Analyze medical imaging samples for a startup MVP (this one was scary) Every one of those jobs started as a quick conversation with a prospect and turned into real invoices because I could say yes. The bottleneck was never capability — it was always cost. When GPT-4o charges north of $10/M output tokens, a single 2,000-token response on a tricky chart costs me about two cents. Multiply by 10,000 images per month and you've got a $200 API line item before you've paid yourself. That's a problem when the whole job is worth $400. So I tested every multimodal model I could find on Global API. Here's the lineup I ended up evaluating. The Contenders Nine models, three providers, one freelanc

Build-in-public. This is the real architecture behind running ~50 local AI agents on 6GB of VRAM — one GPU lock, an eviction watchdog, a resource governor, and a model router. Originally posted on my blog. The question I get most often is some version of "there's no way you run that many agents on a 6GB laptop GPU." The honest answer: not the way you're picturing it. I don't run 50 models at once. I run one model at a time, very deliberately — and most of the engineering is about scheduling, not inference. Here's the actual architecture. The hard constraint: 6GB of VRAM A single consumer GPU with 6GB of VRAM holds roughly one 7B-parameter model at a usable quantization. Two at once? It thrashes — the GPU starts swapping, latency explodes, and eventually a driver out-of-memory can take the whole machine down. I've had the desktop freeze from exactly that. So the first design rule wrote itself: only one heavy model is allowed on the GPU at any moment. That sounds limiting. It isn't — because almost nothing I run is latency-sensitive. A blog post that publishes at 7am doesn't care if it was generated at 6:52 or 6:58. Once you accept that your AI workforce is a batch system, not a chat window, the whole problem changes shape. A lock, not a crowd Every agent that needs the GPU has to take a lock first. It's a simple file-based queue with: FIFO ordering PID-based ownership Stale-lock detection, so a crashed job can't wedge the line forever If an agent can't get the lock within its timeout, it skips gracefully and tries again on its next scheduled run instead of piling up. So at 50 agents, what's really happening is: dozens of cron-scheduled Python workers wake up throughout the day, and the ones that need the model form an orderly line for it. The fleet is huge; the GPU contention is always exactly one. That's the trick. It's less "50 models" and more "50 employees sharing one very busy workstation, politely." Eviction and a VRAM watchdog Even with the lock, idle models l

We are so excited to announce the winners of the Gemma 4 Challenge! This is officially our most...

What flipped in b9437 Build b9437 , published on May 30, 2026 at 20:56 UTC , ships two targeted default-value corrections to llama-bench . Flash attention ( -fa ) shifts from a hard-coded off to auto ( LLAMA_FLASH_ATTN_TYPE_AUTO ), and the GPU-layer count ( -ngl ) changes from the legacy sentinel 99 to -1 . Both values now match what llama-server and llama-cli already used — the bench tool was simply never updated to track them until this build. Quick Answer: Before b9437 (published May 30, 2026) , llama-bench hard-coded -fa off , silently skipping flash attention even on CUDA, Metal, and Vulkan hardware. Build b9437 sets the default to -fa auto and -ngl -1 , matching llama-server and llama-cli . Any pre-b9437 baseline on FA-capable hardware needs a flag-matched re-run to remain valid. PR #23714 , reviewed and merged by maintainers JohannesGaessler and pwilkin, adds the same -fa auto|off|on tri-state flag to llama-bench that the rest of the toolchain already supported. With LLAMA_FLASH_ATTN_TYPE_AUTO as the new default, flash attention activates automatically when the runtime detects a capable backend (CUDA, Metal, Vulkan); on CPU-only hosts it stays off with no error and no output change. Parameter Before b9437 After b9437 Behavioral impact -fa off (hard-coded) auto ( LLAMA_FLASH_ATTN_TYPE_AUTO ) GPU-capable hosts bench with FA active by default; pre/post comparisons require explicit flag-matching -ngl 99 (offload-all sentinel) -1 (runtime decides) CPU-only builds no longer attempt full GPU offload; eliminates spurious CUDA errors when no GPU is present The following verified script (executed successfully, exit 0) demonstrates the behavioral gap in concrete terms — on a capable GPU, the pre-b9437 defaults schedule zero FA rows while b9437 defaults schedule one: def old_llama_bench ( device ): # Before b9437, the default bench matrix used FA=0, so FA rows were skipped. return [{ " device " : device [ " name " ], " ngl " : 0 , " fa " : 0 }] def b9437_llama_bench ( de

Apple is planning to raise prices in response to the ongoing memory shortage. In an interview with The Wall Street Journal, Apple CEO Tim Cook says "price increases are unavoidable:" We're doing our best to mitigate the huge increases that are being passed to us, and we've been trying to shield our customers from the […]

I recently calibrated a recovery-rate model that had only two weak features. Its point accuracy was almost nothing — R² basically zero. I expected its uncertainty estimates to be junk too. They weren't: the 90% conformal prediction intervals covered ~89% of held-out outcomes. Valid, just wide . That surprised me enough to nail it down, because it contradicts a belief a lot of us carry around: "my model isn't accurate, so I can't trust its uncertainty." For split conformal prediction, that's backwards. Here's the precise statement, a runnable demo, and the one caveat that actually bites. Coverage is a property of the procedure, not the model Split conformal prediction gives a distribution-free, finite-sample marginal coverage guarantee : P( Y ∈ Ĉ(X) ) ≥ 1 − α and it holds for any point model, as long as the calibration and test data are exchangeable. The model is a black box. You fit it however you like, then on a held-out calibration set you take the (1−α) quantile of the absolute residuals, and that quantile becomes the half-width of your intervals. Nowhere does that construction require the model to be good. A bad model just has large residuals, so the calibration quantile is large, so the intervals are wide — wide enough to still cover at the stated rate. Accuracy doesn't buy you validity ; it buys you efficiency (narrower intervals at the same coverage). The demo (numbers are reproducible, seed fixed) Same dataset and target, three models from strong to useless, target coverage 90%: model R² marginal coverage mean interval width gradient boosting 0.741 0.895 5.39 weak linear (1 noisy feature) 0.061 0.905 10.39 predict-the-mean −0.000 0.907 10.83 All three land at ~90% coverage. The only thing that changes is width: the good model's intervals are half as wide . That's the whole story in one table — validity is constant, efficiency tracks accuracy. import numpy as np from sklearn.linear_model import LinearRegression from sklearn.ensemble import GradientBoostingReg

In the previous series, when we optimized our neural network, we had to write quite a bit of training code ourselves. First, we created an optimizer object that used Stochastic Gradient Descent (SGD) to optimize final_bias . Then we wrote loops to calculate the derivatives required for gradient descent. We trained the model for up to 100 epochs . For each training example, we: Ran the input through the neural network to get a prediction. Calculated the loss. Calculated the derivatives of the loss function. After processing all three training points, we used: optimizer . step () to take a small step toward a better value for final_bias . Then we used: optimizer . zero_grad () to clear the accumulated gradients before starting the next epoch. All of this required a considerable amount of training code. Let's see how Lightning helps simplify this process. Organizing Training Logic with Lightning Previously, we created a class to store the weights, biases, and the forward() function. The optimization-related code was written separately outside the class. With Lightning, we can keep all of this logic in one place. We start by creating the class as usual, and then add a few new methods. Configuring the Optimizer The first method is configure_optimizers() . def configure_optimizers ( self ): return SGD ( self . parameters (), lr = self . learning_rate ) This method tells Lightning how the neural network should be optimized. The learning rate is stored in the self.learning_rate variable that we defined earlier. Defining a Training Step Next, we add a method called training_step() . def training_step ( self , batch , batch_idx ): input_i , label_i = batch output_i = self . forward ( input_i ) loss = ( output_i - label_i ) ** 2 return loss This method receives: A batch of training data from the DataLoader. The index of that batch. Inside the method, we: Extract the input and label from the batch. Run the input through the neural network. Calculate the loss using the squared r

"How does ChatGPT think ?" It doesn't. The entire mechanism behind every chatbot is almost anticlimactic: it predicts one next word , adds it, and repeats. I built a tiny interactive predictor so you can be the model — and it explains both the magic and the flaws. 🔮 Be the model: https://dev48v.infy.uk/ai/days/day6-next-token.html This is Day 6 of AIFromZero — AI literacy, one concept a day, no code to follow. 1. It only predicts the NEXT word Given everything so far, the model outputs a probability for every possible next word, picks one, appends it, and runs again with the longer text. Paragraphs, code, poems — all of it is this one step on repeat. "the cat sat on the ___" → P(mat) high, P(bird) low 2. It's a probability over the WHOLE vocabulary The output isn't one word — it's a number for every word it knows (100,000+ for a real model). Most are near zero; a handful are plausible. The bars in the demo are that distribution, over a tiny vocabulary. 3. Autoregression: feed the output back in After picking a word, it becomes part of the input for the next prediction. Predict → append → predict again. Because each new word conditions on all the previous ones, short local choices add up to coherent long text. 4. Temperature = the creativity dial Once you have probabilities, how do you choose? Temperature reshapes them before sampling: Near 0: the top word always wins — safe, repetitive. High: the odds flatten, so rarer words get a real chance — creative, error-prone. p = p ** ( 1 / temperature ); // then renormalise and sample Drag the slider in the demo and watch the bars sharpen or even out. That one knob is what an API calls "creativity." 5. Where do the probabilities come from? In my toy, from counting which word followed which in a few sentences (a "bigram" with 1-word memory). A real LLM replaces the counting with a giant neural network trained on much of the internet, and its memory spans thousands of words. The mechanism is identical — only the quality of th