标签：#llm

Claude Opus 4.8 shipped today. The benchmarks are a distraction — here is what actually changes about how your agents run tomorrow. Anthropic announced Claude Opus 4.8 at 16:00 UTC on June 3, 2026. The launch post leads with the usual benchmark deltas: SWE-bench Verified up 4.1 points, GPQA Diamond up 2.9, TAU-bench tool-use up 6.4. There is a chart. There is a marketing line about "the most capable agentic model we have ever shipped." If you stop reading there, you will miss the three things that will change how your production agents behave starting tomorrow. I have spent the morning re-running our internal agent harness against Opus 4.8 and reading the model card line by line. Two of the three changes are improvements. One of them is a silent regression that will bite anyone who pinned the model ID. Here is the full picture. What 4.8 actually changes The model card and release notes ship three changes that the launch blog post does not foreground: Cache-aware routing inside long agentic loops. The 4.7 router treated every tool-call cycle as a fresh planning step. 4.8 keeps an internal trace of which cache breakpoints were hit on the previous step and biases the next plan toward extending those traces. In agent harnesses that already use prompt caching aggressively (Claude Code, the Agent SDK with cacheControl: "ephemeral" on the system prompt), cache hit rates jumped from a measured ~46% on 4.7 to ~71% on 4.8 across a 30-step coding loop. The 200k context window now actually behaves at 200k. Anthropic published a needle-in-a-haystack chart in the model card going out to 200,000 tokens. The 4.7 chart got noticeably worse past ~140k tokens; the 4.8 chart is flat. This sounds like a benchmark thing. It is not. It changes the cost equation for "just stuff everything in context" patterns that 4.7 quietly punished by degrading accuracy. claude-opus-4-7 was not aliased. The launch shipped a new model ID — claude-opus-4-8 — and the previous ID is still callable. But if y

Since the early days of GenAI, when ChatGPT launched in late 2022, we began using prompt engineering to direct chatbots (and later LLMs) with human language instructions to provide us answers to questions or take actions (in a high-level…) In 2025, companies such as OpenAI and Anthropic began releasing a new agentic concept called “AI Agent”, an autonomous system that uses an AI model as its "brain" to perceive an environment, make independent decisions, and execute multi-step tasks using digital tools. Unlike passive chatbots that just answer questions, an agent can plan its own workflow, run commands, and browse the web to achieve a specific goal without constant human supervision. In this blog post, I will explain the concept of Context-as-Code and share some coding examples. Introducing Context-as-Code Traditional prompting is a one-way street. You type out your instructions, send them off, and that text never changes. AI agents operate completely differently. Because they work on their own, every action they take creates a mountain of new data. Every time an agent opens a file, checks an error, or runs a tool, it adds more information to the pile, which quickly overwhelms a standard chat screen. Context-as-Code treats the agent like a stateless compute engine. Instead of a massive text prompt, we use version-controlled files ( CLAUDE.md , AGENTS.md ) to establish structural boundaries, separating the permanent project rules from the temporary, dynamic session memory. Context-as-Code transforms loose AI prompts into version-controlled engineering assets by using structured Markdown files to establish permanent, auditable boundaries directly within a project repository. The Discovery Stage (Onboarding the Agent) Before an agent writes a single line of code, it must parse the overall project layout. These files act as the "map" for an incoming AI. llms.txt Serves as a lightweight text directory mapped out in Markdown format. Placed at the root of a project or webs

I learned this the hard way: when an LLM says “it matches the docs”, it can still be wrong for a boring reason—it didn’t read the part that matters. I’m building a small SaaS (checklists as a service). No users yet. Plenty of documentation already. And at some point my docs stopped being an asset and started turning into a liability. This is the story of how I rebuilt my documentation so that an LLM could actually read it end-to-end —and how that restructure helped me. The moment I got scared: “silent misses” The docset grew. I kept asking the LLM to verify tasks against it. And then I noticed a pattern that felt worse than hallucinations. Not “the model invented stuff”, but “the model confidently said it matches ”—while quietly missing exceptions, prohibitions, and thresholds. Keyword scanning instead of reading. I called it silent drift : code slowly moves away from conventions, while the invariants remain only in my head. In a project with roles, audit, and CI/CD security gates, that kind of drift isn’t “just messy docs”. It’s how you lose the ability to implement and review changes consistently. I couldn’t do it manually (and I couldn’t delegate it fully) I knew I had to redo the documentation. But I also knew I couldn’t realistically do it all by hand. At the same time, I couldn’t just tell an LLM: “Rewrite everything according to approach X.” Not enough context, too easy to lose control. So I went with a third option: build a reliable process out of unreliable components— me + an LLM . Step 1: I separated my docs into domains (and forced the model to actually read) First, I extracted domain areas from the old documentation—the vocabulary I was using to describe the project and its parts. I tried to keep domains mutually independent (so the overall framework stays holdable in my head). Then I ran the same loop for each domain: I asked the LLM to read all old docs carefully and extract requirements for that domain. I moved those requirements into a dedicated fil

A hands-on knowledge-distillation project: Qwen2-VL-7B → 2B for UI-screenshot understanding, trained, evaluated and benchmarked end-to-end on an M4 Pro. 2.4× faster — and why the teacher lost on ROUGE-L.

The dominant production failure mode for LLM agents in 2026 isn't bad reasoning — it's capacity. Here's what the data shows, why nobody demos it, and the capacity-engineering patterns that actually keep agents alive under load.

AI Builder Notes - May 2026 AI-assisted notes from my liked-tweets feed, organized around agent workflows, browser traces, model loops, and guardrails. Practical takeaways Start with the workflow, not the agent. A useful agent task has a source of truth, a narrow action, a verifier, and a stop condition. “Review this repo” is vague. “Find auth bugs in these routes, cite file lines, run the relevant tests, and stop after the first credible exploit path” is a workflow. Use dynamic workflows in claude code - to do the vibe bits for thinking through a workflow. Think of it like this - you can describe in natural language an entire workflow consisting of multiple agents at various steps - I want the docs updated, tests passed, security review done and also playwright tests done. Dynamic workflows figures out which parts can be divided in parallel and what should be done sequentially. Creates a flowchart - and writes JS code for it. Its a JS script that can execute subagents at scale and deterministically [1] Planner/executor split is the way to go. Spend the expensive model on taste, decomposition, and risk discovery. Use cheaper or narrower models for repeatable implementation once the task has tests, rubrics, logs, or examples. [2] Do not judge an agent workflow by the model name alone. If the loop has repo access, a rubric, a way to inspect tool calls, and a verifier, a less fashionable model can still do useful work. The Letta Code / GLM 5.1 review-bot example is interesting for that reason, not because “someone used X instead of Y” is interesting by itself. [3] Prefer small interfaces to giant tool menus. MCP tool call definitions are rotting your context! The monday.com GraphQL example was the clearest cost warning: one task used 15k tokens through SDK/code-mode and 158k tokens through a real MCP server. MCP is useful, but a menu of tools is not automatically an efficient interface. [4] [5] For browser work, save the trace. Run the workflow once, inspect wasted act

Large language models (LLMs) understand and generate text from prompts. OpenAI exposes models through the Responses API . The official openai npm package is the practical way to call it from Node.js. This post covers common patterns beyond a single prompt string. Prerequisites OpenAI account Generated API key Enabled billing Node.js version 26 openai package installed ( npm i openai ) For Markdown output: marked , dompurify , and jsdom ( npm i marked dompurify jsdom ) Client setup Create a client with your API key (read from the environment in production). import OpenAI from ' openai ' ; const client = new OpenAI ({ apiKey : process . env . OPENAI_API_KEY }); The same SDK can target other hosts that implement a compatible API by setting baseURL and apiKey : const client = new OpenAI ({ apiKey : process . env . LLM_API_KEY , baseURL : ' https://your-gateway.example/v1 ' , }); Azure OpenAI uses AzureOpenAI instead. Many third-party gateways support Chat Completions only; the examples below use client.responses.* , so confirm your provider supports the Responses API (especially for tools like web search). Basic integration Pass a string as input and read output_text from the response. const response = await client . responses . create ({ model : ' gpt-5.5 ' , input : ' Write a one-sentence bedtime story about a unicorn. ' , }); console . log ( response . output_text ); System prompt Use top-level instructions for stable behavior (tone, format, role). They take precedence over casual wording in the user message. const response = await client . responses . create ({ model : ' gpt-5.5 ' , instructions : ' Reply in one short sentence. Use plain language. ' , input : ' Explain what an LLM is. ' , }); console . log ( response . output_text ); Few-shot prompting Pass prior turns as an input array with user and assistant roles, then the new user message. Keep task rules in instructions . const response = await client . responses . create ({ model : ' gpt-5.5 ' , instructions :

When I decided to ship this blog in four languages, I had a clear mental ranking. English would win on volume. Spanish would be runner-up because of the sheer speaker count. Japanese would stay steady because it's my native language. Portuguese, I figured, was the long tail. I added it mostly out of completism. Twenty-two days later, the GA4 snapshot disagrees with every part of that ranking. PT: 748 pageviews , 709 sessions EN: 195 pageviews , 176 sessions JA: 27 pageviews , 29 sessions ES: 7 pageviews , 7 sessions That is Portuguese pulling roughly 3.8× English, 28× Japanese, and 107× Spanish on the same blog, same publishing cadence, same author. One Portuguese article on its own (a post about a 24-hour security agent: 375 PV) got more pageviews than my entire English blog combined. I wrote that article hoping Spanish would surprise me. Instead Portuguese surprised me, and Spanish quietly continued to not exist. The setup, so you can discount my numbers properly This is not a clean comparative experiment. It's a single blog, kenimoto.dev , running four language directories ( /en/ , /ja/ , /pt/ , /es/ ). Articles get translated through a cross-language LLM pipeline, then hand-edited for register and locale (BR Portuguese vs PT Portuguese, LatAm-neutral Spanish vs Spain Spanish). The window: 2026-04-30 to 2026-05-21, 22 daily snapshots. EN has 26 articles. JA has 25. PT has 17. ES has 10. So PT has fewer articles than EN and still beats it almost 4 to 1. If you stop reading here, take this one thing: language asymmetry can swallow article-count asymmetry whole . Adding articles in a saturated language is slower than adding articles in an underserved one. Why Portuguese pulled ahead I don't think the answer is "Portuguese readers like me more." I think three asymmetries are stacking on top of each other. 1. TabNews is a community door English doesn't have TabNews is a Brazilian developer community where you can post a technical article and have it actually read by h

Originally published on rikuq.com . Republished here for Dev.to's readers. The FinOps Foundation Framework is the reference architecture for cloud financial management. It's been maintained by the FinOps Foundation (a Linux Foundation project) since 2018 and has matured into the de facto standard most serious cloud cost work is built on. In 2026 it received a substantial refresh that extended its scope from pure cloud spend to include AI/ML, SaaS, licensing, and broader technology categories. For practitioners thinking about formalising FinOps practice — or evaluating providers who claim to do FinOps — knowing what the Framework actually covers is what separates a real implementation from a marketing label. This post walks through the Framework structure, the 2026 updates, and how it applies specifically to AI/ML spend. I'm Ravi. I run three production AI SaaS solo ( Prism , Citare , BatchWise ) and do advisory work on FinOps via rikuq services . The walkthrough below is what I use when teams ask "what does the FinOps Foundation Framework actually look like in practice?" TL;DR Element What it is Phases Three concurrent operational modes: Inform, Optimize, Operate Principles Six foundational principles guiding all FinOps practice Capabilities The functional areas of activity a FinOps practice covers Personas Engineering, Finance, Procurement, Leadership, Operations, ITAM, Sustainability 2026 additions Executive Strategy Alignment, Technology Categories taxonomy, Converging Disciplines recognition AI/ML extension New Technology Category with specifics on GPU/CPU differential, token pricing, make-vs-buy economics The six foundational principles Before the structural mechanics, the Framework's six principles establish the cultural and operational mindset. They're worth knowing because they're how the Framework's authors test whether something is "really" FinOps or just cloud cost cutting. Teams need to collaborate. Engineering, Finance, Procurement, and Business teams w

How to make LLMs deterministic, in plain English. The version I share with founders and product teams before they make decisions worth real money. You use AI tools every day. But can you explain what happens when you hit send? Most people cannot. And that gap is costing them. Bad prompts. Broken products. Decisions made on the wrong assumptions. The Hard Truth Every LLM explainer out there is written for researchers or so basic it tells you nothing useful. Neither helps you build better products or work with AI more effectively. This is the version I share with senior leaders, founders, and product teams before they make decisions worth real money. 1. It Is Not a Search Engine. It Is Not a Database. It Is a Prediction Machine. When you type a prompt and hit send, the LLM is not finding an answer from somewhere. It is predicting the most likely words to follow your input. Based on patterns it learned from billions of documents. That is the whole process. Wrong: "The AI knows the answer." Right: "The AI predicts the most likely answer based on what it has seen." This changes everything about how you use it. When an AI gives you a wrong answer confidently, it is not broken. It is doing exactly what it was built to do. Predict. Not verify. 2. The Autocomplete Comparison (And Why It Only Gets You Halfway) You have probably heard the phrase "autocomplete on steroids." It is not wrong. But it misses something important. Your phone autocomplete learned from your messages. An LLM learned from most of the written internet. Books. Research papers. Code. Billions of examples. At that scale, the patterns start to look a lot like real thinking. Not because the model understands in the way you do. Because it has seen so much that it can predict what a good answer looks like. When I was building AstroNayak I fed Vedic astrology principles into the system prompt. The LLM produced interpretations that genuinely surprised me. It did not know Vedic astrology. It had seen enough of it t

The row looked perfect. rating: 7 . Valid JSON, right type, no nulls, no missing keys. My schema check waved it through. The page had returned HTTP 200. The selectors hadn't moved. Everything green. A rating of 7 on a 5-star site is impossible. The model invented it, formatted it correctly, and handed it to me with total confidence. That's the failure I want to talk about. Not the scraper that breaks loudly. The one that hands you a clean-looking row that is quietly, plausibly false — and sails past every check you have, because your checks are all looking at the shape of the data, and the lie is in the value . TL;DR HTTP 200, intact selectors, and valid JSON tell you the form is fine. They say nothing about whether the value is true. When an LLM extracts from messy free-text, structured-output mode guarantees you get valid JSON. It does not guarantee the content is real. The model fills uncertain fields rather than leaving them empty — because the schema demands a complete row. A ~60-line value-level sanity gate (ranges, dates, cross-field, reference, language) catches the obvious lies before they hit your database. Real code and real output below. The honest catch: this gate catches rule violations , not plausible lies inside the allowed range . A rating: 4 where the truth is 2 slides right through. I'll be specific about where the gate stops. Two different ways a scraper lies to you I wrote about source drift last week — the case where the page changes underneath you and a 30-line schema check catches the structure shifting. That's an input problem. The source mutated; your agreement with the page broke; you detect it by watching the shape. This is the other end of the pipe. The source is fine. The page is intact, the selectors are correct, the structure is exactly what you expected. The thing that lied to you is the model , on the extraction step, when you asked it to pull structured fields out of a paragraph of human prose. Those two failures feel similar and t

In This Article The Deal Intelligence Gap Data Model Design Transcript Analysis with Claude Automated Follow-Up Drafting PostgreSQL JSONB Storage Putting It Together The Deal Intelligence Gap Most CRM systems are excellent at storing what happened — call logged, email sent, stage updated — and poor at capturing what was learned. A sales call produces qualitative intelligence that is genuinely valuable for deal strategy: what objections surfaced, how strongly the prospect signaled interest, what next steps were agreed to, and what risk flags the conversation revealed. That intelligence almost never makes it into the CRM because it requires someone to spend 15 minutes synthesizing unstructured notes into structured fields. Large language models change this equation. Given a call transcript, Claude can extract structured deal intelligence in seconds — categorizing sentiment, identifying specific objections, recommending stage movement, and flagging risk signals — with accuracy that equals or exceeds what a well-trained sales analyst would produce manually. Data Model Design The data model centers on two tables. The deals table stores core deal attributes as a JSONB column, which allows flexible schema evolution without migrations as the intelligence fields change over time. The deal_activities table records each interaction — calls, emails, meetings — with the raw content in TEXT and the extracted intelligence in a separate JSONB column. A GIN index on both JSONB columns enables fast attribute queries across the deal pipeline. CREATE TABLE deals ( id UUID PRIMARY KEY DEFAULT gen_random_uuid (), company TEXT NOT NULL , contact TEXT , stage TEXT , attributes JSONB DEFAULT '{}' , created_at TIMESTAMPTZ DEFAULT now (), updated_at TIMESTAMPTZ DEFAULT now () ); CREATE TABLE deal_activities ( id UUID PRIMARY KEY DEFAULT gen_random_uuid (), deal_id UUID REFERENCES deals ( id ), activity_type TEXT , raw_content TEXT , intelligence JSONB , created_at TIMESTAMPTZ DEFAULT now () )

About this series. I'm going to take a fresh paper - Self-Distilled Agentic Reinforcement Learning (SDAR, arXiv:2605.15155 ) - and architect it end to end on AWS: the system design, the actual gate code, the evaluation plan, and a brutally honest cost model. What I'm not going to do is wave a benchmark number around. Reproducing a paper like this costs thousands in GPU time, and I'd rather show you the machinery than a screenshot you can't audit. The design is the deliverable. This is Part 1. A small, infuriating problem Picture an LLM agent working a web-shopping task. It reads the goal, searches, clicks a category, filters, opens a product, compares, adds to cart - twelve steps in all. At the end, it bought the wrong thing. So you do what reinforcement learning tells you to do: you score the trajectory. Reward = 0. Bad agent. Now answer this: which of the twelve steps was actually wrong? Maybe step 3, the search query, was fine and step 9, a filter choice, doomed everything. Maybe steps 1–11 were brilliant and step 12 fat-fingered the wrong button. Your single scalar reward has no idea. It punishes all twelve equally, including the eight that were correct. That's the supervision problem in agentic RL, and it's the thing this whole series is about. Why "just use RL" isn't enough for agents RL has become the default way to post-train LLM agents. The catch is that the reward usually lands at the trajectory level - one number for the entire multi-step episode. For a single-turn task ("answer this question"), that's tolerable; the action and the outcome are close together. For a long-horizon agent - ten, twenty, fifty turns of searching, calling tools, and reacting to an environment - it's a disaster of credit assignment . The signal is too coarse to tell the model which decisions earned the reward and which torpedoed it. You can throw more episodes at it and let statistics sort the credit out eventually. But "eventually" on a 30-turn task burns a lot of expensive comp

Token Budgeting: Optimizing Generative AI Costs and Performance Modern generative AI applications offer unprecedented capabilities, yet their operational costs can quickly escalate. The primary driver of these costs, alongside computational resources, is token consumption . Understanding and implementing effective token budgeting strategies is not merely an optimization; it is fundamental to building scalable, efficient, and economically viable AI systems. The Economics of Tokens Tokens are the atomic units of text that large language models (LLMs) process. Whether you're sending a prompt (input tokens) or receiving a response (output tokens), each token incurs a cost. This cost varies by model, but the principle remains: more tokens mean higher expenses and often, increased latency due to longer processing times. Efficient token management directly impacts your application's bottom line and user experience. Strategic Pillars of Token Efficiency Optimizing token usage requires a multi-faceted approach, focusing on both input and output, as well as the underlying model choices. 1. Input Optimization: Crafting Smarter Prompts The most direct way to save tokens is to be judicious with the information sent to the model. Every word in your prompt counts. Concise Prompt Engineering : Avoid verbose instructions or unnecessary conversational filler. Get straight to the point. Instead of: "Hey AI, I was wondering if you could please help me summarize this really long article I have here. It's about quantum computing. Could you make it brief, maybe just a few sentences?" Opt for: "Summarize the following article about quantum computing in three sentences: [Article Text]" This significantly reduces input tokens without sacrificing clarity. Context Window Management : LLMs have a finite context window , the maximum number of tokens they can process at once. Sending an entire document when only a specific section is relevant is wasteful. Employ techniques like: Summarization : P

Notes on Serving LLMs with TensorRT-LLM and Triton 2026-05-31 · LLM serving / NVIDIA stack These are working notes on taking an open-weights LLM from a Hugging Face checkpoint to a production-style serving endpoint on the NVIDIA stack — TensorRT-LLM for the engine, Triton Inference Server for the deployment surface — and benchmarking it honestly against vLLM on multi-GPU hardware. They follow the harness in trtllm-triton-serving (4× H100, NVLink). The goal is to move from "I use vLLM" to "I can stand up the NVIDIA inference stack on real multi-GPU hardware and reason about the trade-offs." 1. The serving pipeline The path from checkpoint to endpoint has four stages. Each one is a place where a decision affects latency, throughput, or accuracy: Checkpoint — a Hugging Face model. Engine build — compile to a TensorRT-LLM engine for a fixed tensor-parallel degree, precision, and batching policy. Model repository — wrap the engine in a Triton tensorrt_llm -backend model repo. Serving + load test — trtllm-serve (or Triton) exposes an OpenAI-compatible endpoint; a load generator drives it under controlled concurrency. The key mental shift from vLLM: TensorRT-LLM does ahead-of-time compilation . vLLM is a runtime that takes the model and serves it; TensorRT-LLM builds an engine specialized to your GPU, TP degree, and precision first. That build is where the performance comes from, and also where the rigidity comes from. 2. Tensor parallelism (TP) For a model that doesn't fit on one GPU — or to cut latency — TensorRT-LLM shards each layer across GPUs. On a 4× H100 NVLink box, TP=4 means every forward pass does an all-reduce across the four GPUs over NVLink. The all-reduce is not free. On this fabric it tops out around 77 % of the NVLink budget (see the separate NVLink-wall notes ). For prefill (large tensors) you're bandwidth-bound and TP helps. For decode (one token at a time) you're pinned against the small-message latency floor, and past a point more TP makes decode slowe

0 % vs 50 %: making a RAG agent refuse to hallucinate 2026-05-31 · LLM / RAG A retrieval-augmented agent is only as trustworthy as its behaviour on questions whose answer isn't in the corpus . The failure mode is quiet: instead of saying "I don't know," the model invents a confident, well-formed, wrong answer. This post shows a single guardrail that takes that from common to never — and, crucially, measures it. Reference architecture: nim-agent-blueprint — agentic RAG on the NVIDIA NIM stack with a built-in eval harness. The ablation The agent loop is plan → retrieve → generate → validate . The interesting variable is the generation prompt's contract with the retrieved context: Configuration Out-of-corpus hallucination rate Generate freely from context ~50 % Guarded prompt (answer only from context; otherwise abstain) 0 % Same model, same retriever, same questions. The only change is a prompt that makes "I can't answer that from the provided sources" a first-class, rewarded output — plus a validate step that checks the answer is grounded in retrieved spans before returning it. On in-corpus questions, retrieval recall@3 stayed at 94–100 % , so the guardrail buys safety without costing coverage. Why "just prompt better" isn't the lesson The lesson isn't the prompt — it's that the difference between 50 % and 0 % is invisible without an eval harness . A demo that only asks in-corpus questions looks perfect in both configurations. You only see the 50 % when you deliberately ask things the corpus can't answer and score groundedness . So the blueprint ships with: retrieval hit-rate (is the answer even retrievable?), answer groundedness via LLM-as-judge (is the answer supported by what was retrieved?), latency , and OpenTelemetry traces per agent step. That's the difference between "it works on my five questions" and "here is the number a partner can hold me to." Takeaway For enterprise RAG, abstention is a feature, not a failure. Make "I don't know" a rewarded output, vali

Now that almost everyone has thought about or is actively integrating AI workflows into their projects, some might ask is this all worth the cost? Many think the current economics of the AI space don't scale and that there will be upward price movement. Others still might not be comfortable with sending their data to remote services for processing. Then there is the crowd that wants to deploy models in small spaces with limited compute. Are there ways we can deploy small models locally and run at a lower cost? Yes with Knowledge Distillation . Knowledge distillation can get a bad rap due to it's questionable use in training some Large Language Models (LLMs). But it's a perfectly valid way to transfer performance from a larger model to a smaller one. Especially when both models are yours and/or open. This article will explore progressive distillation which is a technique to incrementally transfer knowledge from a series of larger teacher models into a smaller student. Install dependencies Install txtai and all dependencies. pip install txtai [ pipeline - train ] datasets Setup the Training Pipeline The first step we need to do is setup up the training pipeline. We'll use the Hugging Face Training framework to build a series of models. The following code establishes a train method, test method and loads the classification training data. from datasets import load_dataset from transformers import AutoModelForSequenceClassification , AutoTokenizer from txtai.pipeline import HFTrainer , Labels def train ( teacher , student , distillation , ** kwargs ): trainer = HFTrainer () model = AutoModelForSequenceClassification . from_pretrained ( student , trust_remote_code = True ) tokenizer = AutoTokenizer . from_pretrained ( student , trust_remote_code = True ) return trainer ( ( model , tokenizer ), ds [ " train " ], columns = ( " sentence " , " label " ), maxlength = maxlength , teacher = teacher , distillation = distillation , ** kwargs ) def test ( model ): labels = Labels (

"A language model that answers questions is a tool. A language model that decides which questions to ask and then acts on the answers is something else entirely." Introduction: When Models Started Deciding For the first several years of modern NLP, the task was always the same: given input, produce output. One forward pass. One completion. Done. In 2022, a paper from Google Brain asked a different question. What if, instead of producing an answer directly, a model could reason about what information it needs, act to retrieve it, and revise its thinking based on what it found? The paper was ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2022). Applying it to an LLM created something qualitatively different: a model that could take real-world actions and adapt its reasoning based on what came back. A completion model is a calculator. An agent is a process: it has a goal, takes steps toward it, and updates when things go wrong. This week I went deep on the architecture behind these systems, the frameworks that define them, and what the open problems look like from a research perspective. Part 1: What Makes a System "Agentic"? The word "agent" gets used loosely in current literature. A clean definition comes from Russell and Norvig's Artificial Intelligence: A Modern Approach : An agent is anything that perceives its environment through sensors and acts upon that environment through actuators. For an LLM-based system, this is a loop: perceive an observation, reason about what to do, act via a tool call or output, observe the result, and loop again. But not every loop qualifies as agentic. Three properties distinguish genuinely agentic systems from tool-augmented chatbots: Property What It Means Goal persistence Maintains the original goal across multiple steps without re-prompting Adaptive planning Revises its approach based on intermediate results Tool autonomy Decides when and which tools to use, not just how to use one it was told to call Mos

The Question Nobody Wants to Ask: Does Your Skill Actually Help? You spent an afternoon crafting a carefully structured Skill for your agent. Clear steps, thorough edge-case notes, well-formatted output requirements. You tested it manually a few times, the outputs looked great. You shipped it. Three weeks later, you notice that some task success rates have gone down compared to before the Skill existed. This is not a hypothetical. In May 2026, Microsoft Research published two concurrent papers — SkillLens ("From Raw Experience to Skill Consumption") and SkillOpt ("Executive Strategy for Self-Evolving Agent Skills") — that measured this failure mode at scale. Their finding: negative transfer happens in 25% of cases , and you cannot reliably identify the bad skills just by reading the text. One paper answers "why skills sometimes backfire." The other answers "how to make skills systematically better." Together they sketch a new paradigm for agent capability improvement. Part One: SkillLens — Mapping the Full Skill Lifecycle A Skill Is Not a Point — It's a Pipeline Most practitioners think of a Skill as "a block of text instructions for an agent." SkillLens decomposes this into a three-stage lifecycle : Stage 1: Experience Generation Target model M runs training tasks, producing an experience pool of trajectories (both successes and failures) ↓ Stage 2: Skill Extraction Extractor model E distills the experience pool into a structured skill document — procedural knowledge under a fixed budget ↓ Stage 3: Skill Consumption The same target model M, equipped with the extracted skill, is evaluated on held-out test tasks Notice there are two distinct roles in this chain: the Extractor (distills knowledge from trajectories) and the Target (consumes knowledge to improve task performance). SkillLens's central insight is that these two roles are independent — a strong task executor is not necessarily a strong extractor, and vice versa . Two New Metrics: EE and TE To separate thes

How RAGScope Knows Which Chunks Your LLM Actually Used Your retriever fetched 10 chunks. Your LLM only used 3. RAGScope shows a precision score of 30 out of 100. The question every new user asks: how does it know? There is no OpenTelemetry attribute that says "this chunk was in the context window." RAGScope infers it — and the way it does this is the most consequential piece of engineering in the whole tool. There Is No "In Context" Attribute in OTel The OpenTelemetry semantic conventions for generative AI ( gen_ai.* ) define attributes for model, input/output tokens, and retrieved documents. They do not define anything like gen_ai.chunk.reached_llm or gen_ai.retrieval.used_document_ids . When your RETRIEVER span fires, you get a list of documents. When your LLM span fires, you get a prompt and a completion. The two spans are connected by a parent-child trace relationship — but there is no attribute that maps which retrieved documents appear in which prompt. This gap matters. A reranker might drop 7 of your 10 chunks. Your application code might apply a token budget and truncate 4 more. From the trace alone, you cannot tell. RAGScope needs this information to compute the precision sub-score — the highest-weighted metric at 40% of the overall score. Getting it wrong would make precision meaningless. The Substring Match — How assembleContext Works RAGScope's answer is in src/enrichment/pipeline.ts , in a function called assembleContext : function assembleContext ( chunks : RagChunk [], llmSpans : ParsedSpan []): RagChunk [] { const llmPrompts = llmSpans . map (( s ) => s . prompt ). filter (( p ): p is string => !! p ); if ( llmPrompts . length === 0 ) return chunks ; let position = 0 ; return chunks . map (( chunk ) => { if ( ! chunk . content ) return chunk ; const inContext = llmPrompts . some (( p ) => p . includes ( chunk . content ! )); if ( inContext ) { return { ... chunk , inContext : true , contextPosition : position ++ }; } return { ... chunk , inContext :