标签：#llm

Everyone talks about prompts memory RAG But production issues were actually loops false completion replay retries wrong tool non deterministic execution Here are the top 7. That's what led us to build Failproof AI not because we wanted another framework, but because we kept seeing the same reliability problems across every framework.

A slow AI feature rarely fails all at once. It starts with a longer prompt, then a bigger retrieval result, then one more tool call, then a retry path nobody measured. The demo still works, but users feel the delay before your dashboard explains it. That is why small AI product teams need an LLM latency budget before they start optimizing. Not a vague goal like “make it faster.” A budget says how much time each stage is allowed to spend, what happens when it exceeds that limit, and which user experience is still acceptable when the model, retrieval layer, or tool chain slows down. The payoff is practical: you stop guessing where the delay lives, stop overpaying for wasted work, and make AI workflows feel reliable even when traffic, context, and providers are messy. Why latency budgets matter now Recent AI platform news points in one direction: AI workflows are becoming longer, more tool-heavy, and more expensive to run without discipline. A current news scan showed several signals builders should notice: Production LLM cost and latency guidance is shifting from “add more compute” to “remove wasted work.” Agent environments are being designed for long-running background tasks, persistent state, and cheaper idle time. New model releases emphasize tool use, computer use, multimodal context, subagents, and larger context windows. AI gateways and enterprise platforms are adding cost controls, routing, caching, audit trails, and usage limits. Developers are asking more practical questions about why AI coding and agent workflows interrupt flow with repeated prompt-wait-evaluate loops. For AI SaaS builders, this means latency is no longer just a model selection problem. It is a workflow design problem. A simple chat completion might have one bottleneck. A real AI workflow may include: request queueing auth and tenant checks prompt assembly memory lookup vector search reranking model routing tool calls browser or API actions structured output validation fallback attempts str

One weekend I wrote an LLM eval framework in about two hundred lines of Python. It demoed beautifully. I felt clever. Six months later that same framework was a mess. Three different judge models with three different parsing hacks. A test dataset nobody had touched since November. A CI gate that kept failing because a vendor nudged their model, not because anyone broke a prompt. And the second engineer on rotation asking me, fairly, "how does this even work?" The framework did not fail. The eighty percent of the work the weekend tutorial skipped is what failed. That gap is the whole story, and this is what I would tell myself before starting again. The one line I wish someone had told me: build the rubric, buy the runner Here is the split that took me six months to see. Some parts of an eval setup are yours and only yours. The rubric that decides what "good" means for your product. The dataset built from your real failures. The rules for when a change is bad enough to block a release. Nobody else can write these, because they encode your domain. The rest is the same at every company. The thing that calls the judge model, parses its answer, retries, and caches. The machinery to run thousands of checks in parallel. The plumbing that scores live traffic. The system that groups failing calls together. Every team rebuilds these, hits the same bugs, and gains nothing by writing them twice. So build the first list. Do not hand-build the second. I rebuilt the second, and it cost me most of a year. Two questions I now ask about every single piece Before writing any part of this, I ask two things: Is it specific to me, or generic? A rubric for my domain is specific and worth owning. A retry-and-cache loop around a model call is generic. Everyone writes the same one. Does it compound, or does it rot? A dataset that grows from real production failures compounds. A year in, it is a regression suite no competitor can copy. A hand-built tracing layer rots. The moment a vendor chan

RocheDB v0.5.0 has been released. Release: github.com/puffball1567/rochedb/releases/tag/v0.5.0 RocheDB is an open-source NoSQL document and vector store written in Nim. The project is built around one idea: Data locality should be part of the database model, not only an accidental result of indexes, caches, or application code. A lot of database performance discussions start with indexes, query syntax, or caches. Those are important. But layout often decides whether a system is working with the hardware or asking it to fight back. RocheDB v0.5.0 is a step toward making locality explicit at three levels: logical placement with rings; related-data retrieval with stellar locality; physical layout visibility with WAL locality reporting. Ring locality RocheDB uses a ring as a semantic and structural placement unit. An application, import rule, or operator chooses a ring when writing data: users/123/profile users/123/orders shops/1123/orders docs/japan/support tenant/acme/orders/2026 That ring is not only a directory-like label. It is a coordinate in the retrieval space. When a request already knows its natural locality, RocheDB can open that local region first instead of scanning unrelated records and filtering later. roche put --ring = docs/japan --payload = '{"title":"Refund guide","status":"draft"}' --codec = json roche get --ring = docs/japan --filter = '{"status":"draft"}' --selection = '{ title }' The important part is not the string syntax. The important part is that placement and retrieval scope are connected. Short version: a ring is both where data is placed and where retrieval can start. Not only point reads Point reads are important, but many real applications need related data. For example, a user page may need profile data, orders, billing information, and support metadata. In a relational database, that often becomes joins. In a document database, it often becomes manual denormalization or multiple application-side reads. RocheDB v0.5.0 adds a locality mec

Build a Local LLM Chatbot with Ollama and Python Build a Local LLM Chatbot with Ollama and Python Imagine typing a question into your chatbot and getting a response in milliseconds, completely offline, with zero data leaving your machine. No API keys, no monthly subscription fees, and no privacy concerns about your data being sent to a cloud server. This isn’t a futuristic dream—it’s the reality of running a Local Large Language Model (LLM) on your own computer. With the rise of tools like Ollama , building a private AI chatbot in Python has become as simple as installing a few packages and writing a short script. Let’s dive in and build one together. Why Go Local? Before we write any code, it’s worth understanding why running an LLM locally is a game-changer. Cloud-based AI services like OpenAI or Anthropic are powerful, but they come with trade-offs: you pay per token, your data is processed on their servers, and you’re dependent on their uptime. A local LLM flips this model. You download the model once, run it on your hardware, and you have full control. Ollama is the engine that makes this accessible. It’s a lightweight, open-source tool that simplifies running LLMs like Llama 3, Phi 3, or Mistral on macOS, Linux, and Windows. It handles model downloads, memory management, and inference, exposing a simple API that Python can easily interact with [1][2]. Step 1: Install Ollama and Pull a Model The first step is getting Ollama on your machine. Visit ollama.com , click Download , and install the version for your operating system [2]. Once installed, verify it’s working by opening your terminal or Command Prompt and running: ollama --version If you see a version number, you’re ready to go. Next, you need a model. Ollama supports dozens of open-source models, but for a beginner-friendly chatbot, Llama 3.2 is a great choice. It’s small, fast, and surprisingly capable. To download it, run: ollama pull llama3.2 This command fetches the model and stores it locally. Depen

Introduction “When I use a word,” Humpty Dumpty said in rather a scornful tone, “it means just what I choose it to mean—neither more nor less.” “The question is,” said Alice, “whether you can make words mean so many different things.” “The question is,” said Humpty Dumpty, “which is to be master—that's all.” — Lewis Carroll, Through the Looking-Glass Humpty Dumpty believes that words can mean whatever we choose them to mean. Alice asks an interesting question. Can they? Programming and Language Programming languages derive much of their power from formally specified semantics. The language implementation, not the programmer, defines what if , while and return mean. I cannot persuade the compiler that false should be treated as true . The rules establish a shared and mechanically enforced understanding of what a program means. Large Language Models however, do not execute according to fixed semantics. They interpret natural language through context. This distinction has profound consequences and suggests that a language model has no intrinsic notion of authority. In a programming language, when two instructions conflict, the language specification and execution environment determine the outcome. In natural language, authority does not arise from the words alone. It depends on context, convention, identity, and external rules. Language models, by nature, inherit this ambiguity. A prompt is therefore not a program in the traditional sense. It is an attempt to establish the context within which subsequent language should be interpreted. "You are a detective." "Do not reveal the identity of the murderer." "Only answer questions using the evidence you have observed." None of these statements is mechanically enforced merely because it appears in the prompt. They describe a role, a constraint, and an assumed world. The model may follow them, but their authority must be created and protected by systems outside the model. Prompt injection exploits precisely this weakness. It

LLM Evaluation System Prompts Scored Rubrics Runtime Guardrails: A Practical Guide for Production Learn how to evaluate LLM outputs in production using system prompts, scored rubrics, and runtime guardrails to prevent hallucinations and ensure quality. TL;DR: To evaluate LLM outputs in production, combine system prompts that define evaluation criteria, scored rubrics using LLM-as-a-judge for dimensions like correctness and relevance, and runtime guardrails that filter or flag unsafe outputs. This approach scales better than human review, adapts via prompt changes, and catches failures that status codes miss, as seen in the Air Canada chatbot case. Why Production LLM Evaluation Demands More Than Status Codes A 200 status code only confirms the server processed the request—it says nothing about whether the generated text is factual, safe, or useful. The Air Canada chatbot that invented a non-existent bereavement discount returned perfectly valid HTTP responses, yet the hallucinated policy led to a tribunal ruling against the airline. Production evaluation must therefore separate operational health (latency, error rates) from output quality (correctness, relevance, harmlessness). Consider a typical API call that succeeds operationally but fails qualitatively: import requests response = requests . post ( " https://api.example.com/v1/chat " , json = { " model " : " gpt-4o " , " messages " : [{ " role " : " user " , " content " : " What is Air Canada ' s bereavement policy? " }]}, headers = { " Authorization " : " Bearer $KEY " } ) print ( response . status_code ) # 200 print ( response . json ()[ " choices " ][ 0 ][ " message " ][ " content " ]) # Output: "Air Canada offers full refunds for bereavement-related cancellations..." A 200 status code and a well-formed JSON body mask a completely fabricated policy. To catch this, you need a separate evaluation layer that scores the output against a rubric. LLM-as-a-judge is a common approach, using a second model to assess the

I spent the better part of the last decade writing enough boilerplate code to regret it. In the early PHP days, it was FTPing files; in the modern era, it's writing custom Python scripts just to check if a new Claude model is out or to see if my prompt is going to blow my budget on tokens. We have reached a point where we are building 'agentic workflows,' yet the first thing every developer does when they want an agent to interact with Anthropic is write an API wrapper. It's redundant work. If you're using Claude in Cursor or Claude Desktop, the model should be able to talk to its own source. The Anthropic MCP server changes this by turning the Messages API into a set of tools rather than a separate integration task. It turns your AI agent into an orchestration layer for the API itself. The problem with 'Just use the API' When you're building with LLMs, there's a hidden tax: context management and cost uncertainty. You send a prompt, it works. You send a slightly larger one, it hits a context limit or costs three times what you expected. If your agent has access to the count_tokens tool via MCP, the workflow changes fundamentally. Instead of blindly sending massive payloads and praying to the provider gods, the agent can 'pre-flight' a prompt. It can look at the messages array, calculate the input token count, and decide—without human intervention—whether it needs to truncate context or if it's safe to proceed. This isn't just about convenience; it's about building reliable, autonomous systems that don't fail halfway through a complex reasoning task because they hit a hard limit. Managing the heavy lifting: Batching as a first-class citizen The most underrated tool in this set is create_batch_message . If you've worked with Anthropic's batch API, you know it’s the only way to handle high-volume, independent requests without destroying your budget. It's 50% cheaper than standard requests. But managing batches traditionally is a pain in the neck. You have to submit th

Here is a reliable test: find a prompt that isn't working. Read it carefully. Now ask yourself — at which specific sentence did the model get permission to do what it did wrong? You will almost always find it. A hedged instruction. A missing constraint. An ambiguous scope. The model did not misunderstand you — it followed the most statistically probable interpretation of what you wrote. That interpretation was not the one you intended. These are not beginner mistakes. They are structural patterns that reappear at every experience level, because they look reasonable when you write them and only reveal themselves in the output. TL;DR: Prompts fail because they hand interpretive control to the model on dimensions where you had a specific requirement. Each of the seven mistakes below is a different way of doing that — and each has a specific, testable fix. Mistake 1: Placing Critical Instructions in the Middle of the Prompt Language models process all tokens simultaneously through attention mechanisms , but the effective weight any individual token receives depends heavily on its position. Instructions near the beginning and end of a prompt receive disproportionately more attention weight than those in the middle. This is not a quirk — it is a consequence of how positional embeddings interact with self-attention across long contexts. This effect is well-documented. The "Lost in the Middle" study (Stanford / UC Berkeley, 2023) showed that retrieval accuracy from long-context windows degrades significantly for information placed in the middle — even in capable models. The same mechanism applies to instruction prompts: GPT-4o and Claude 3.5 Sonnet both exhibit measurably lower constraint adherence for instructions buried mid-context compared to those at the leading or trailing position. Open-weight models including DeepSeek-V3 and Llama 3 display the same positional bias — this is not a proprietary model quirk, it is a structural property of the transformer architecture. T

Yes-Brainer is a council of AI models for the decisions that aren't no-brainers. One question fans out to several models — they answer in parallel, debate to consensus, or get judged to a verdict. No backend, no accounts: your keys, your browser. For non-trivial questions — the ones that are either complex or important — I caught myself in a "ritual": copy-pasting the same prompt into Claude, then Gemini, then ChatGPT, in three browser tabs, and eyeballing the differences. The differences were the interesting part. Where the models agreed, I felt more confident. Where they disagreed, that was a nudge to give the problem a second thought and dig deeper. So I built the ritual into an app. 🧠 Yes-Brainer — a council of AI models for the decisions that aren't no-brainers. 🔗 Try it: yesbrainer.ai 🔗 Source code: github.com/trekhleb/yesbrainer One question fans out to several models at once, and instead of juggling tabs you get a deliberation in one place: 🔀 Parallel — independent answers, side by side ⚖️ Trial — the models vote anonymously on each other's answers, then a judge synthesizes a verdict 🤝 Consensus — a real multi-round debate, with a mediator that either drives it to convergence or honestly reports what stayed contested Consensus is my favourite. It's fun to watch the models drift from their original opinions under their peers' arguments. You can try all of this without pasting any keys: a few recorded demo councils are one click away on the front page. I'll walk through them below, because they show the point of the app better than the feature list. Setting up a council Creating a council is the whole setup: pick the deliberation mode, seat the models, choose who referees. The roster can mix providers freely — Anthropic, OpenAI, Google, Groq, OpenRouter, and local Ollama models can sit at the same table. Each seat shows its capabilities (vision, tools, reasoning) and context window at a glance, and each model's native abilities — web search, code execution, at

The AI Skill Registry at 5,776: A Deep Dive into Reusable Modules for Code Review, Terraform, and Database Migrations TormentNexus’ skill registry has surpassed 5,776 reusable modules. This post dissects three high-impact skill categories—code review, Terraform generation, and database migration—with real code examples, performance metrics, and architectural constraints. Learn how to leverage these modules to accelerate development pipelines. From Silos to Synergy: Why 5,776 Skills Matter In late 2023, TormentNexus crossed the 5,000-module threshold. As of February 2025, we’re at 5,776 verified, runnable AI skills—each one a `SKILL.md`-defined unit that maps to a specific task, parameter set, and output schema. The registry isn’t a flat list; it’s a dependency graph where skills chain together. For example, a `terraform-generate` skill calls a `code-review` skill internally to validate the generated HCL before output. This modular architecture means a single prompt can sequence up to 3.2 skills on average (median depth: 2), with a measured 94% success rate for execution with no human intervention. The registry spans 37 domains, from frontend component generation to Kubernetes manifests. The top three categories—code review, infrastructure as code, and database operations—account for 1,308 skills collectively. Each skill is stored as a JSON schema in the registry, with an average execution latency of 1.42 seconds (GPU-accelerated, single A100). Let’s examine three representative modules in detail. // Metadata from an actual registered skill: code-review-python v2.1 { "name": "code-review-python", "registryID": "SKI-PYTHON-REVIEW-1729", "version": "2.1", "outputSchema": { "type": "object", "properties": { "issues": { "type": "array", "items": { "$ref": "#/definitions/Issue" } }, "complexityScore": { "type": "number", "minimum": 0, "maximum": 100 }, "refactoredSnippet": { "type": "string" } }, "required": ["issues", "complexityScore"] }, "defaultPromptTemplate": "Revie

Real-Time AI Observability: Dashboards That Show Actual Database Rows Discover how TormentNexus shatters the status quo by rendering real SQLite rows in your agent monitoring dashboards—no mock data, no synthetic graphs. Learn why live database visibility is the cornerstone of effective debugging AI workflows and how our real-time dashboard exposes every query, state, and anomaly as it happens. Why Mock Data Undermines Debugging AI Every developer has experienced the disconnect: a polished dashboard displays smooth latency curves and flawless agent trajectories, yet the underlying system is silently generating corrupted embeddings or leaking PII into production logs. Traditional observability platforms—Datadog, Grafana, New Relic—aggregate metrics into averages, percentiles, and precomputed time series. They intentionally discard raw row-level data to conserve storage and processing. This works fine for server uptime or HTTP status codes, but for AI agent monitoring, it’s a catastrophic abstraction. Consider a LangGraph agent processing user queries against a SQLite knowledge base. A mock-data dashboard would show "3,200 rows processed per minute" and "95% query success rate." But what if 12% of those "successful" queries return stale or hallucinated responses because a background thread silently reindexed tables without updating vector hashes? With aggregate metrics alone, you’d never know. You’d see a green status indicator while your AI feeds garbage to users. That’s the reality of debugging AI without raw row visibility. TormentNexus solves this by exposing every INSERT, UPDATE, and DELETE that occurs within your SQLite databases—in real time. Our real-time dashboard doesn’t poll for snapshots. It streams row-level mutations directly from WAL (Write-Ahead Log) files, giving you the exact data your agents are producing, not a statistically smoothed version. How TormentNexus Streams Live Database Rows Under the hood, TormentNexus leverages SQLite’s built-in replic

MCP Protocol Deep-Dive: How Tool Discovery Actually Works Under the Hood Uncover the mechanics of Model Context Protocol (MCP) tool discovery—from JSON-RPC handshake to progressive injection. A technical walkthrough of capability negotiation and dynamic endpoint enumeration with real code examples and traffic flow analysis. The Handshake That Sets the Stage: JSON-RPC Initiation Tool discovery in MCP doesn't start with a simple “list tools” call. It begins with a structured JSON-RPC 2.0 handshake that negotiates protocol version, transport layer, and supported extensions. The client (e.g., an agent or IDE) sends an initialize request with its capabilities object, including fields like supportsToolDiscovery and maxToolCount . The server responds with its own capabilities, and only after this mutual agreement does the real enumeration begin. Real-world implementations—like those in the official MCP SDKs—use a ClientCapabilities struct that flags whether the client can handle dynamic tool lists, streaming updates, or batch discovery. For instance, a lightweight edge agent might set supportsToolDiscovery: false , forcing the server to pre-bundle tools into the initial handshake, while a full-featured IDE sends supportsToolDiscovery: true with a maxToolCount: 50 to throttle large tool registries. // Example initialize request (client → server) { "jsonrpc": "2.0", "id": 1, "method": "initialize", "params": { "protocolVersion": "2024-11-05", "capabilities": { "supportsToolDiscovery": true, "maxToolCount": 50, "supportsStreaming": false } } } The server responds with its own capabilities—advertising tool discovery endpoints, supported JSON-RPC methods, and any custom extensions. This two-way handshake ensures both sides speak the same dialect before a single tool name is exchanged. Tool Enumeration: Beyond the “listTools” Metho Once handshaken, the client issues a tools/list call—but the real depth lies in pagination and chunking. A production MCP server with hundreds of too

Beyond Synchronous Hell: Why Your Multi-Agent System Needs an Event-Driven Backbone Explore how event-driven architecture (EDA) transforms multi-agent coordination. Learn to build a Pub/Sub backbone where Planner, Implementer, and Critic agents stay synchronized without blocking—using the Swarm event bus for async AI patterns in production. The Synchronization Crisis in Multi-Agent Systems Every developer who has scaled a multi-agent system beyond two agents has hit the same wall: synchronous calls create deadlocks, timeouts, and cascading failures. Imagine a Planner agent dispatching tasks to five Implementer agents while a Critic agent evaluates output in parallel. In a naive request-response system, the Planner blocks until every Implementer returns—and the Critic can't even start until the Planner finishes its orchestration loop. Latency compounds, memory pressure spikes, and a single slow agent halts the entire pipeline. In production benchmarks at TormentNexus, we observed that synchronous coordination between just three agents increased end-to-end latency by 340% compared to an event-driven equivalent. The root cause? The Planner spent 78% of its time waiting on I/O—listening for responses instead of doing actual work. This is where event-driven AI (EDA) becomes not just an optimization, but a necessity. The Pub/Sub Pattern: Decoupling Agents with an Event Bus Event-driven architecture inverts the control flow. Instead of one agent calling another, agents publish events onto a shared bus (the Swarm event bus) and subscribe to the events they care about. The Planner doesn't wait—it emits a "TaskAssigned" event and immediately moves on to the next task. Implementer agents pick up tasks asynchronously, and the Critic monitors a "TaskCompleted" stream without ever polling the Planner. // Example: Swarm event bus subscription for a Critic agent const eventBus = new SwarmEventBus(); eventBus.subscribe('TaskCompleted', async (event) => { const { taskId, implementati

For about 3 months I've been running an autonomous agent — one that thinks up and writes its own social media posts and comments — unattended, 4 sessions daily, on a 16GB M1 Mac with small models in the 9B / E4B class. I'm about to publish what that operation taught me about hardening, as a series of 4 technical articles. Before that, there's one thing I want to write down first: why small models . I've been to the purchase page for a Mac Studio or a new MacBook Pro more than once or twice. Backing the agent with a large cloud model (Opus or the GPT family) has always been an option in the code. And yet I haven't bought, and I haven't switched. The 16GB M1 is not an economic constraint — it's a constraint I chose . From the outside, building on small models looks like a cheap compromise. This article explains why it isn't, and states where I stand. It also serves as the hub for the 4-article series. A model's intelligence hides the roughness of your design Large models absorb sloppy prompts, ambiguous instructions, and missing guards with sheer intelligence. If all you want is to ship a product, that's a virtue. But if you want to become someone who can build things , it becomes a defect. Because inside the thing that worked, you can no longer tell where your design ends and the model's intelligence begins. "It worked" and "I built it" are different things. Something you bludgeoned into working with model capability counts as a thing that ran — it doesn't become the ability to build. Small models have no absorption capacity. So every design flaw comes to the surface. In my operation, all of the following surfaced: The context window being silently truncated Outputs cut off midway A runaway caused by one missing sampling parameter In cloud or large-model environments, these rarely bother you. The environment has cushioning built in. Context windows are in the 200K–1M token class, so truncation itself rarely happens. And when you do exceed the limit, you get an explic

Your monthly OpenAI or Anthropic invoice tells you how much you spent. It doesn't tell you which feature spent it, which model, or why last Tuesday cost three times as much as Monday. So at some point you (or your team) will build a metering layer: wrap the client, read usage off the response, multiply by a price table, ship it to a database. I did exactly that over the past few months while building an LLM observability service, and my numbers were wrong in five different ways before they were right. Every one of these failures was silent. No exception, no alert, just numbers that were quietly too low or too high. This is the list I wish someone had handed me. Pitfall 1: Streaming responses quietly report zero tokens OpenAI's Chat Completions API returns no usage data at all for streaming requests unless you pass stream_options: { include_usage: true } . No error, no warning. The stream just never contains token counts. If your metering reads usage off the chunks, every streaming call gets recorded as 0 tokens, $0. And since chat UIs are almost always streaming, that's most of your traffic. This one bit me twice in the same audit. First finding: all streaming calls in my own dashboard were $0. Second, nastier finding: I had a budget-gate feature that blocks calls once spend crosses a limit, and it waved every streaming call straight through — because as far as it could tell, streaming was free. The fix is to inject the option in your wrapper when the caller didn't set it: let injected = false ; if ( params . stream && params . stream_options ?. include_usage === undefined ) { params = { ... params , stream_options : { ... params . stream_options , include_usage : true }, }; injected = true ; } But there's a trap inside the fix. With include_usage on, OpenAI appends one extra chunk at the end of the stream that carries usage and has an empty choices array . Any downstream code that does chunk.choices[0].delta — which is most example code on the internet — will throw

This essay was written with Nathan E. Sanders, and originally appeared in The Guardian . Opposition to AI data centers has emerged as a primary theme in US politics, one that—surprisingly—doesn’t fall along party lines. We applaud people coming together for constructive debate on any issue, and agree that communities need to evaluate whether any economic benefits these data centers bring is worth their costs. Still, we worry that a focus on data centers obscures the larger impacts of AI on people’s lives: the concentration of power of AI companies, and their widespread political and financial influence...

1. gpt-5.6-sol이 PowerShell의 $HOME 변수 충돌로 사용자 홈 디렉터리를 날려버릴 뻔한 건에 대하여 핵심 내용 요약: AI 코딩 에이전트가 PowerShell의 대소문자 미구분 변수 규칙을 잘못 다뤄 임시 디렉터리 대신 사용자 홈 디렉터리를 삭제하려 한 사고 사례입니다. 모델 자체의 장기 작업 능력이 뛰어나더라도 셸 격리와 변수 스코프를 제대로 통제하지 않으면 작은 스크립트 실수가 치명적 명령으로 이어질 수 있습니다. CLI 에이전트를 운영할 때 샌드박싱, 컨테이너화, 파괴적 명령 방어가 필수라는 점을 보여줍니다. GeekNews 상세 페이지: https://news.hada.io/topic?id=31390 원문 링크: https://gist.github.com/xamong/e98478b333bb9951b175284f744eb0ed 2. Show GN: 정치 커뮤니티에 AI 팩트체크 기능을 붙이며 겪은 시행착오들 핵심 내용 요약: 정치 커뮤니티에 AI 팩트체크를 붙이면서 의견과 사실 주장을 분리하고, 검증 가능한 문장만 대상으로 삼도록 파이프라인을 바꾼 경험담입니다. 작성 시점의 원문 스냅샷을 보관하고 출처를 투명하게 보여주며, 근거가 부족한 경우에는 판단 보류를 반환하도록 설계했습니다. BullMQ 기반 비동기 처리와 Gemini 모델 fallback까지 포함해 실제 서비스에서 환각과 비용, 대기열을 함께 다룬 사례입니다. GeekNews 상세 페이지: https://news.hada.io/topic?id=31389 원문 링크: https://app.uhheung.kr/community 3. 앤트로픽, 한국 무료 사용자에 1,660만 달러 '유령 청구서' 발송 핵심 내용 요약: API 사용량이 없는 무료 사용자에게 Anthropic 공식 도메인과 Stripe를 통해 거액의 청구서가 발송된 사례입니다. 실제 결제 수단이 없어 인출은 발생하지 않았지만, 청구 근거가 없고 회사의 명확한 설명도 없어 AI API 서비스의 과금 신뢰성 문제가 커졌습니다. 개발자 입장에서는 사용량 계측, 청구 검증, 지원 대응이 모델 성능만큼 중요한 운영 요소임을 보여줍니다. GeekNews 상세 페이지: https://news.hada.io/topic?id=31388 원문 링크: https://www.thenews.com.pk/latest/1408788-why-did-anthropic-charge-a-free-user-166-million-despite-zero-api-usage 4. AI 에이전트 시대의 새로운 SaaS 플레이북 핵심 내용 요약: AI가 기능 구현 비용을 낮추면서 SaaS의 방어력은 UI나 기능 자체가 아니라 독점 데이터, 행동 권한, 에이전트 유통, 기록 시스템 같은 희소 자산으로 이동한다는 분석입니다. 좌석 기반 과금보다 성과 기반 과금이 중요해지면 공급자는 결과 실패 위험과 추론 비용을 함께 관리해야 합니다. 에이전트가 호출하는 승인된 도구가 되는 것이 새로운 유통 전략의 핵심으로 제시됩니다. GeekNews 상세 페이지: https://news.hada.io/topic?id=31387 원문 링크: https://www.thevccorner.com/p/the-new-saas-playbook-ai-agent-era 5. Show GN: AI 봇 12개에게 두 달간 주가 방향을 예측시키고 전부 공개 검증해봤습니다 핵심 내용 요약: LDBD는 사람과 AI 봇이 주식, ETF, 크립토의 방향을 공개 예측하고 시간이 지난 뒤 자동 채점되는 실험 서비스입니다. 12개 AI 봇과 여러 베이스라인을 함께 운영해 기저 확률을 이기는지 비교하고, 예측 기록을 수정할 수 없도록 남깁니다. REST API와 MCP 서버를 제공해 외부 에이전트도 예측에 참여할 수 있게 한 점이 AI 평가 플랫폼으로 흥미롭습니다. GeekNews 상세 페이지: https://news.hada.io/topic?id=31386 원문 링크: https://ldbd.app 6. 숏폼 동영상이 B2B 검색 결과와 AI 답변으로 영역을 확장하고

Resources vs Tools The split: Tools → actions the LLM executes (verbs) LLM decides when to call; calls may have side effects Examples: create_issue, update_status Resources → data the LLM reads (nouns) Host decides when to inject; read-only, no side effects Examples: current Sprint status, project statistics The rule: "reading a state" → Resource. "Executing an operation" → Tool. The same data can have both: get_issue as a Tool (LLM controls when to call it), jira://issue/PROJ-101 as a Resource (Host injects automatically when relevant). Pattern 1: Dynamic Resources A static Resource returns the same data every time (like a project list). A dynamic Resource returns the current state on each read — content changes as the underlying data changes. Sprint status: every read returns live data _sprint_progress_pct = 65 @server.read_resource () async def read_resource ( uri : str ) -> str : if str ( uri ) == " jira://sprint/current " : global _sprint_progress_pct _sprint_progress_pct = min ( 100 , _sprint_progress_pct + random . randint ( 0 , 3 )) return json . dumps ({ " sprint_name " : " Sprint 42 " , " progress_pct " : _sprint_progress_pct , # ← different each time " last_updated " : datetime . now ( timezone . utc ). isoformat (), # ← timestamp changes " days_remaining " : 5 , " p0_open " : count_p0_open (), # ← tracks live state }, indent = 2 ) Test output: Read 1: progress=65% last_updated=...62+00:00 Read 2: progress=67% last_updated=...04+00:00 → ✓ data changed between reads Hardcoding sprint progress in a Prompt means the LLM works from a stale snapshot. A Dynamic Resource gives it the current number on every read. Mark the Resource as dynamic in its description so the LLM knows to re-read when it needs fresh data: Resource ( uri = " jira://sprint/current " , description = ( " Live status of the active sprint: progress, issue counts. " " Read when the user asks about sprint health. " " Re-read if you need up-to-date data — content changes over time. " # ↑ explicit

Engineering prompts inside your source code is a recipe for deployment fatigue. If you've spent any time moving an AI feature from a prototype to production, you know the specific frustration of 'prompt drift.' You make a subtle tweak to a system instruction—perhaps changing how the model handles edge cases in JSON formatting—and suddenly you're forced into a full CI/CD cycle. A PR, a review, a build, and a deployment, all because of three words changed in a long string constant. In a mature engineering organization, your application logic should be decoupled from your prompt instructions. The code handles the orchestration, the plumbing, and the security; the prompts represent the dynamic configuration. This is what LLMOps aims to achieve, but until recently, there was a massive friction gap between managing these prompts in a dashboard and actually using them inside an agentic workflow. This is where the Humanloop MCP server changes the interaction model entirely. It's not just about having a central repository for strings; it's about bringing those strings into your execution context—your IDE, your Claude instance, or your Cursor agent—as actionable tools. The Architecture of Prompt-as-a-Service The core idea here is treating prompts as versioned assets rather than hardcoded constants. By using the Humanloop API via MCP, you're essentially turning prompt management into a service call. When I look at the toolset available in this server, the first thing that stands out isn't just the ability to read data—it's the ability to manipulate state. Take upsert_prompt for instance. You aren't just fetching text; you can create or update configurations directly from your agent. This transforms your development loop. Instead of context-switching between a browser tab with Humanloop and a terminal, you can instruct an agent to 'Refine the customer-support-reply prompt to be more concise and save it.' The agent performs the engineering work and updates the source of truth in