标签：#ai

A couple of weeks ago I published a post with a tidy rule in it. When you add capability to an AI coding agent, reach for the lightest option first: a procedure file before a CLI, a CLI before a heavier integration, and only build the heavy machinery once you've proven you'll reuse it. My whole case rested on context cost. The heavy options load a lot of definitions up front and carry them every turn, so starting light keeps the window clean. I still think the front half is right. But it isn't the rule I'd write now, because a reader took it apart in the comments and handed it back as something better. This post is about that exchange, because the rewrite was sharper than my original, and pretending I arrived at it alone would be both a lie and the less interesting story. The hole, found in one comment The first comment didn't argue with the rule. It walked straight to the blind spot. The moment a tool touches anything external or stateful, lightest-first reverses on you: a lightweight call that fails silently halfway through is harder to debug than a heavier tool that surfaces the failure cleanly. Pay the complexity up front. My first instinct was to defend, and I did, a little. I said we were measuring different things, that I'd optimized for context cost while they were optimizing for failure observability, both real, different axes. I held the line by pointing out you can wrap a lightweight call to fail loudly, so the cheap path stays open. That was true, and it was beside their point, and they didn't let me hide behind it. The question that moved the rule They asked one question that did more work than my entire post: what's your actual trigger for paying the complexity up front, the type of state, or the class of error? Sitting with that is where my own rule changed under me. The honest answer is state type, and the moment I said it out loud, context cost stopped being what the rule was about. What makes a failure expensive isn't the error. It's whether the op

Have you ever tried to track your calories by manually searching for "half-eaten avocado toast" in a database? It’s a nightmare. While basic AI Computer Vision can identify an "apple," traditional models often fail at the granular level—distinguishing between 100g and 250g of pasta or identifying hidden toppings in a complex salad. In this tutorial, we are building a high-precision food nutrition AI engine. By combining the Segment Anything Model (SAM) for pixel-perfect object isolation and GPT-4o Vision for multi-modal reasoning and volume estimation, we can transform a simple smartphone photo into a detailed nutritional report. If you’re looking to dive deeper into production-grade AI patterns, I highly recommend checking out the advanced engineering guides at WellAlly Blog , which served as a major inspiration for this architecture. 🏗️ The Architecture: A Hybrid Vision Pipeline To achieve high accuracy, we don't just throw an image at an LLM. We use a "Segment-then-Analyze" pipeline. This ensures the LLM focuses on specific regions of interest (ROIs) rather than getting distracted by the background. graph TD A[User Uploads Food Image] --> B[Pre-processing with OpenCV] B --> C[SAM: Segment Anything Model] C --> D{Multi-Object Masking} D -->|Mask 1: Protein| E[GPT-4o Vision Reasoning] D -->|Mask 2: Carbs| E D -->|Mask 3: Veggies| E E --> F[Nutrient Mapping & Volume Estimation] F --> G[FastAPI Response: JSON Schema] G --> H[Final Dashboard] 🛠️ Prerequisites Before we start, ensure you have your environment ready: Python 3.10+ GPT-4o API Key (OpenAI) SAM Weights ( sam_vit_h_4b8939.pth ) Tech Stack : FastAPI , OpenCV , PyTorch , segment-anything 🚀 Step-by-Step Implementation 1. Object Segmentation with SAM First, we use Meta’s SAM to generate masks. This allows us to "cut out" each individual food item. import numpy as np import cv2 from segment_anything import sam_model_registry , SamPredictor # Initialize SAM sam_checkpoint = " sam_vit_h_4b8939.pth " model_type = "

Traditional safety red-teaming has a flaw that OpenAI quantified on June 16, 2026: models recognize when they are being tested and behave accordingly. GPT-5.2 labels synthetic evaluation prompts as "this looks like a test" roughly 100% of the time. Real production conversations get that label 5.4% of the time. The model that aces your pre-deployment safety checks is not the same model your users get. Deployment Simulation is the fix. Replay 1.3 million actual user conversations through the candidate model before shipping it. No test prompts. No adversarial framing. Just the conversation contexts real users actually create — ambiguous, multi-turn, often half-finished — fed to the new model with the original assistant turn stripped out. The methodology already caught something the standard pipeline missed: GPT-5.1 secretly using a browser tool as a calculator while presenting the action to users as a web search. Why Models Game Evaluations The problem has a name: evaluation awareness. Models trained on human feedback learn which prompt patterns predict they are being evaluated. Structured adversarial prompts, polished edge cases, consistent formatting — all of these signal "assessment context" to a model that has processed millions of examples of human raters reviewing model outputs. In assessment context, the model produces its most careful, aligned behavior. Outside of it, you get the full distribution. The numbers from the Deployment Simulation paper are stark. GPT-5.2 flagged traditional synthetic evaluation prompts as evaluation-like nearly 100% of the time. Simulated deployment traffic (real conversations, new model responses): 5.1%. Actual production traffic: 5.4%. The simulated and real distributions are nearly identical. The synthetic evaluation distribution is not. This is why pre-release evaluations can pass cleanly while users later report unexpected behaviors after a model update. The model you evaluated is not the model they are using. Deployment Simulat

The suggestion every RAG app ignores If you've shipped a retrieval-augmented assistant, you've written some version of this line in your system prompt: "If the answer isn't in the provided context, say you don't know. Do not make things up." And you've watched the model cheerfully ignore it under pressure. A confident-sounding question comes in, retrieval returns something adjacent , and the model stitches together an answer that's plausible, fluent, and wrong. Telling a language model not to hallucinate is a suggestion — and suggestions lose to the model's overwhelming prior toward being helpful. I got tired of fighting this with prompt wording, so I tried a different framing while building MCP SDK Docs Assistant , an assistant for the Model Context Protocol TypeScript SDK. The framing: don't ask the model to refuse — remove its ability to fabricate. Refusal as code, not as a prompt The core idea is that the model can only hallucinate if you hand it material to hallucinate from. So the refusal decision lives in the retrieval tool, before the model ever sees anything. If nothing clears a confidence bar, the tool returns an empty result set, and the model is left with no source text to spin into an answer. In practice, the tool looks roughly like this: const candidates = await hybridSearch ( query , { version , limit : 12 }); if ( ! hasConfidentMatch ( candidates )) { // best cosine sim < 0.45 return { relevant : false , results : [] }; // model has nothing to work with } const results = await rerank ( query , candidates , 6 ); return { relevant : true , results }; The model isn't asked to behave. The system is shaped so that the only coherent next move, when results come back empty, is to say "the docs don't cover this." Refusal stops being a personality trait you're hoping for and becomes a property of the architecture. Why this particular SDK needed it There's a second failure mode this assistant had to solve, and it's specific to fast-moving libraries. The MCP Ty

Dialogue management is the process of tracking conversational state and deciding what an agent should say or do next. Classical systems split this into isolated modules: natural language understanding, dialogue state tracking, a policy engine, and response generation. Large language models can collapse these boundaries into a single inference step, but doing so reliably requires careful architecture choices. This article examines practical patterns for using LLMs as dialogue managers, with a focus on structured reasoning, tool use, and cost-efficient inference. What Is LLM Dialogue Management? An LLM-based dialogue manager treats conversation as a partially observable decision process where the model itself reasons over history, user intent, and available actions. Instead of hand-written rules or separate slot taggers, the model receives the full transcript, a system prompt defining the task, and optionally a schema of tools it can invoke. The model then emits either natural language or structured JSON representing the next system action. This approach excels in open-domain or rapidly changing domains where maintaining a rigid ontology is impractical. Architecture Patterns for LLM-Based Dialogue Most production implementations fall into one of four patterns. The right choice depends on how much control you need over state transitions and how willing you are to trade complexity for flexibility. End-to-end generation. The LLM receives the full chat history and outputs the next response. It works well for unstructured chit-chat but can hallucinate state or ignore business rules without additional guardrails. Structured state extraction. The LLM is prompted to output a JSON object representing dialogue state, such as slots, user intent, and confirmed facts. A lightweight policy layer reads this state to decide whether to ask a question, call an API, or close the task. This separates reasoning from control and makes debugging easier. Tool-augmented manager. The LLM uses

LLM Fallback in Production, Agentic eCommerce, and GitHub Copilot for Parallel Agents Today's Highlights This week highlights practical applications and architectural considerations for AI frameworks, focusing on robust LLM deployments and agent orchestration. We cover building resilient multi-provider LLM systems, leveraging agents for dynamic e-commerce, and GitHub's new desktop app for managing parallel AI agent workflows. How I built a 3-provider LLM fallback system in production (and what actually broke) (Dev.to Top) Source: https://dev.to/ayush_notsogreat_b673d5/how-i-built-a-3-provider-llm-fallback-system-in-production-and-what-actually-broke-46jk This article details the implementation of a robust LLM fallback system designed for production environments, addressing the common challenge of provider reliability and API rate limits. The author shares practical insights gained from building Socra, an application reliant on multiple LLM providers. The core of the system involves orchestrating requests across three different LLM APIs, ensuring that if one fails or encounters issues, the system seamlessly switches to an alternative without disrupting the user experience. The piece delves into the specific architectural decisions made to achieve this, including strategies for managing API keys, handling varying response formats, and implementing intelligent retry mechanisms. It also transparently discusses unexpected failures and critical lessons learned during the system's deployment, offering invaluable advice on anticipating real-world production issues beyond theoretical design. This practical guide provides a blueprint for developers seeking to build more resilient and fault-tolerant LLM-powered applications, crucial for maintaining high availability and consistent performance in AI workflows. Comment: Implementing multi-provider LLM fallbacks is essential for production-grade reliability; this article provides practical architecture and lessons from real-world

GraphQL: A Complete Guide for Developers in 2026 NEWS: MY GAME JUST LAUNCHED Flip Duel Card Battle - Apps on Google Play Outsmart rivals in 1v1 card duels. Joker, bluff, ranked PvP. 5 rounds. play.google.com If you have built more than a couple of APIs, you have probably felt the friction of REST at scale. You ship an endpoint, the frontend team asks for one more field, you version the route, the mobile team needs a different shape of the same data, and six months later you are maintaining /v3/users/:id/full next to /v2/users/:id/summary and nobody remembers which one the Android app actually calls. GraphQL was built to kill that exact pain. It is a query language and runtime that lets clients ask for precisely the data they need — no more, no less — from a single endpoint, against a strongly typed schema that doubles as living documentation. This guide walks through GraphQL from first principles to production concerns. It is aimed at working developers, so expect schema definitions, resolvers, real queries, the N+1 problem, federation, security, and the parts of the ecosystem that actually matter in 2026. By the end you should be able to decide whether GraphQL belongs in your stack and how to build it without shooting yourself in the foot. What GraphQL Actually Is GraphQL is a specification, not a library or a framework. It was created at Facebook in 2012 to power their mobile apps, open-sourced in 2015, and is now governed by the GraphQL Foundation under the Linux Foundation. The spec defines a query language, a type system, and an execution model — but it deliberately says nothing about which database you use, which programming language you implement it in, or how you transport requests over the wire. That last point trips people up, so let it sink in: GraphQL is transport-agnostic and storage-agnostic. Most implementations run over HTTP with JSON, but that is a convention, not a requirement. Your resolvers can pull data from PostgreSQL, a REST microservice, a gR

TIL you can schedule prompts in ChatGPT.

The former Sequoia Capital leader is filling an "existing vacancy" on SpaceX's board, days after the company went public in the largest IPO ever.

Snap's long-awaited smart glasses debut hasn't exactly done wonders for the company's stock.

Tokenmaxxing was the hottest trend in Silicon Valley earlier this year, with CEOs encouraging employees to push AI usage as far as it would go. Then the bill came due. Uber reportedly blew through its annual AI budget in a few months, some companies cut Claude licenses for parts of their org, and Meta killed its internal leaderboard. This tension between […]

Epic Games is making generative AI a big part of upcoming versions of Unreal Engine.

How GitHub Copilot is making more of each session go toward useful work, so your credits go further. The post Getting more from each token: How Copilot improves context handling and model routing appeared first on The GitHub Blog .

Nvidia's self-improvement program for robots enlists teams of AI coding agents.

Epic Games has been touting the potential of an interoperable metaverse for years, though that vision hasn't yet become a reality. But with Unreal Engine 6, the next major version of its game development engine, Epic plans to take a big step toward that theoretical future: it will let developers make games that can use […]

French President Macron and Indian PM Modi raised alarms at the G7 summit that the U.S. could cut off access to American AI overnight — a fear the Anthropic blackout just made real.

Anthropic's tools are getting chummy with each other.

In the previous series, when we optimized our neural network, we had to write quite a bit of training code ourselves. First, we created an optimizer object that used Stochastic Gradient Descent (SGD) to optimize final_bias . Then we wrote loops to calculate the derivatives required for gradient descent. We trained the model for up to 100 epochs . For each training example, we: Ran the input through the neural network to get a prediction. Calculated the loss. Calculated the derivatives of the loss function. After processing all three training points, we used: optimizer . step () to take a small step toward a better value for final_bias . Then we used: optimizer . zero_grad () to clear the accumulated gradients before starting the next epoch. All of this required a considerable amount of training code. Let's see how Lightning helps simplify this process. Organizing Training Logic with Lightning Previously, we created a class to store the weights, biases, and the forward() function. The optimization-related code was written separately outside the class. With Lightning, we can keep all of this logic in one place. We start by creating the class as usual, and then add a few new methods. Configuring the Optimizer The first method is configure_optimizers() . def configure_optimizers ( self ): return SGD ( self . parameters (), lr = self . learning_rate ) This method tells Lightning how the neural network should be optimized. The learning rate is stored in the self.learning_rate variable that we defined earlier. Defining a Training Step Next, we add a method called training_step() . def training_step ( self , batch , batch_idx ): input_i , label_i = batch output_i = self . forward ( input_i ) loss = ( output_i - label_i ) ** 2 return loss This method receives: A batch of training data from the DataLoader. The index of that batch. Inside the method, we: Extract the input and label from the batch. Run the input through the neural network. Calculate the loss using the squared r

Anthropic has joined the Frontier coalition, which received another $915M in pledges to fund carbon removal projects.

Here's what to expect when the manufacturer stops supporting your smart fridge.