标签：#llm

Automating workflows has always been a priority for me, especially for repetitive and error-prone manual processes. Recently, integrating AI capabilities into these automations offers a great opportunity for those, like me, who seek practical solutions. However, this integration often requires coding or complex API integrations. This is where "low-code/no-code" tools like n8n come into play. I used n8n to set up AI-powered agent flows on my own servers, without compromising data privacy and control. In this post, I will share my experience and explain how to do it step-by-step. Why n8n and Self-Hosted AI Automation? A few years ago, I experimented with different automation tools, especially for simple data transfers and notification flows. But when AI capabilities became involved, I either found them too expensive or avoided cloud-based solutions due to data security concerns. Especially in a client project where we needed to set up an automation handling sensitive financial data, a self-hosted solution became inevitable. n8n offers a wide range of integrations and, thanks to Docker support, I can easily host it on my own server. For me, self-hosting is not just about cost advantage; it also means having complete control over my data. Especially when using AI agents, the question of how much of the data you send to LLMs is logged or used for training is always a concern. With an n8n setup under my control, I minimized these worries. Moreover, n8n's flexible structure gives me the freedom to add as many custom integrations or LLM providers as I want. This is a significant advantage for someone like me who enjoys testing different LLMs. ℹ️ Data Privacy Priority Especially when working with sensitive data, it's crucial to carefully review the data usage policies of cloud-based AI services. Self-hosted n8n offers more control in this regard, making it my preferred choice for critical workflows. Setup Steps: Quick Start with Docker Compose The easiest way to run n8n on y

Local Inference Powers Browser Sign Language, Open-Source Agent Infra, & AI Engineering Guides Today's Highlights This week highlights practical advancements in local AI, featuring a browser-based sign language reader running entirely on-device, new open-source infrastructure for building and evaluating AI agents, and a comprehensive guide to AI engineering from scratch, focusing on building and shipping models efficiently. I Built a Webcam Sign-Language Reader in the Browser (No Cloud) (Dev.to Top) Source: https://dev.to/dev48v/i-built-a-webcam-sign-language-reader-in-the-browser-no-cloud-11hg This article details the creation of a real-time sign language reader that operates entirely within a web browser, without relying on cloud services or model uploads. The developer showcases how to achieve genuinely useful AI functionality, traditionally associated with heavy research labs and GPU clusters, using client-side processing. This approach emphasizes privacy, reduced latency, and accessibility by making advanced AI applications runnable on consumer hardware, specifically within the browser environment. The implementation leverages lightweight models optimized for on-device inference, demonstrating the power of WebAssembly or WebGPU for local execution of machine learning. Such a system offers significant advantages for applications requiring immediate feedback or handling sensitive user data, aligning perfectly with the principles of local AI and empowering developers to deploy sophisticated multimodal solutions without external dependencies. This project serves as an excellent example of practical, self-hosted AI and multimodal processing on consumer hardware. Comment: Running a vision model this complex purely client-side with decent performance is impressive. It really pushes the boundaries of what's feasible for local, privacy-preserving multimodal AI in the browser. trycua/cua — Open-source infrastructure for Computer-Use Agents (GitHub Trending) Source: https

A solo developer with a $200/month budget can now access the same AI coding power that cost enterprises $50,000/month just two years ago. The secret isn't one tool — it's knowing how to mix and match three different access models to get frontier output at budget prices. I've been running this exact stack for months. Here's the breakdown. The Three Ways to Access AI Coding Models Before we talk strategy, understand your three options. Each has a wildly different cost profile. Option 1: Self-Hosted Open Models With models like GLM-5.2 hitting near-Claude Opus quality under MIT license, self-hosting is finally viable. The math is straightforward. Hardware cost: A dedicated GPU server (RTX 4090 or A100) runs $300–$800/month. An H100 rental starts at $1.99/hour on platforms like RunPod. Break-even point: According to cost analysis from multiple providers, self-hosting becomes cheaper than APIs at roughly 5–10 million tokens per month for premium-tier models [1]. Below that volume, you're paying for idle hardware. The catch: You need DevOps skills. Model deployment, quantization, monitoring, failover — it's real infrastructure work. If you save $500 on compute but burn out managing GPUs on weekends, you lost money. Best for: Teams with predictable, high-volume workloads and existing DevOps capability. Think 100M+ tokens/month where savings hit $5M+ annually [2]. Option 2: Pay-Per-Token APIs The default starting point. You pay exactly for what you use. Current pricing (early 2026, per 1M tokens): GPT-4o: $2.50 input / $10.00 output Claude 3.5 Sonnet: $3.00 input / $15.00 output Gemini 1.5 Pro: $1.25 input / $5.00 output DeepSeek V3: $0.27 blended (yes, really) Together AI (Llama 70B): $0.88 blended [1] The pricing floor crashed when DeepSeek V3 arrived at $0.27/M tokens with GPT-4-class quality. Open-source models routed through providers like Together AI or Cerebras ($6–12/M tokens at 969 tok/s) give you more options than ever. The trap: Pricing scales linearly forever. A

A new repository-level dataset, published on GitHub under CC0-1.0, helps researchers and developers discover multilingual developer content across READMEs, issues, and pull requests. The post Accelerating researchers and developers building multilingual AI with a new open dataset appeared first on The GitHub Blog .

AI Builder Notes - Week of June 14, 2026 My thoughts and my twitter’s feeds thoughts This week was all about the ‘loop’ and Fable . The Loop The best way I can describe it is: design the flowchart. Think of the deterministic flowchart on how you want your agents to work. Aim to have: more deterministic bits - this keeps things more predictable more verification bits - this is agent feedback more agent tool calls - this, on a frontier LLM, makes it perform better. The ‘loop’ is essentially: goal -> agent acts -> verifier checks -> state/memory updates -> policy decides next action -> repeat/stop/escalate now the specific implementation of this - will differ based on what you’re working on. Fable Fable capabilities are absolutely insane, I tried it myself and it is entirely worth it for you to spend 2 minutes looking at this. There are a few projects that I fire up a new model into to see what’s it gonna do. A project I wanted to build was a way to teach and demonstrate ‘spin’ in table tennis, every frontier model before Fable fumbled hard. But Fable outshined them with ease: https://srijanshukla.com/artifacts/spin-lab/ If you personally did not experience a big shift in capability, you are probably not asking it a complex enough or ambitious enough task. Fable came, and Fable was taken away. The United States Government(USG) was reported with a jailbreak or sorts - which Anthropic considers not significant. The USG anyway banned Fable just after few days of release. Big drama. Fable was very pricey $$$$ Hence, people developed some patterns of work on those few golden days of Fable being available. - use Fable as planner/architect/taste/spatial/front-end judge. - use GPT-5.5/DeepSeek/Kimi as executor/worker. Other things Openrouter released their Fusion feature as a model on their platform, accessible via API. Fusion is basically council-of-LLMs pattern - providing results that can rival the frontier Fable 5 solo. Google Open Knowledge Format - https://github.com/Goo

When I shipped Trooper , a privacy-aware LLM proxy written in Go, I didn't have a marketing plan. I had GitHub traffic analytics and a habit of checking them obsessively. Seven weeks later, I have something more useful than a viral moment: a ranked table of every traffic spike, what caused each one, and proof that the exact same playbook that worked at launch still works when you have something new to say. What is Trooper? Trooper sits between your app and your LLM provider. When your cloud quota runs out, it automatically falls back to a local Ollama instance with zero code changes on your end. It also tracks session context, so your agents don't go blind between calls. It's not a chatbot wrapper. It's plumbing. Which makes the distribution story more interesting, because plumbing doesn't go viral the way demos do. The Data GitHub gives you 14-day rolling windows for clones and views. I screenshotted them obsessively and tracked every spike. Here's the full ranked table: Rank Date Clones Unique Cloners Views Unique Visitors Driver 🥇 1 May 13 375 173 1,113 ~140 Reddit wave peak 🥈 2 May 10-12 312 137 974 133 Reddit launch spike 🥉 3 Jun 10 289 124 749 101 "Escalate the model" r/ollama post 4 Jun 11 268 112 840 95 Decaying from Jun 10 spike 5 Jun 12 240 99 739 74 Decaying from Jun 10 spike 6 Jun 9 175 102 802 100 Organic 7 Apr 25 174 71 664 113 Early Reddit posts 8 Jun 7 171 110 876 110 Organic recovery 9 Jun 6 163 104 755 102 Organic recovery 10 May 29-30 122 73 610 83 LinkedIn post 11 May 25 76 48 495 53 Claude Code integration chat What I learned 1. Reddit is the only thing that moved the needle, and community fit matters more than size The #1 and #2 peaks were both Reddit-driven. On May 10-11, I posted across r/ollama, r/LocalLLM, r/ClaudeCode, and r/Gemini simultaneously. Total views across those posts: ~7,000. r/ollama alone drove nearly 4,000 of those views. Not r/LocalLLM. Not r/ClaudeCode. r/ollama , the smallest of the four communities. The reason: Trooper so

On June 9, Anthropic shipped Claude Fable 5 — the most capable coding model the industry had ever seen. Three days later, the U.S. government ordered it offline for every user on Earth . No warning. No transition period. One directive, and the frontier vanished overnight. 📖 Read the full version with charts and embedded sources on ComputeLeap → The same week, Z.ai (Zhipu AI) released GLM-5.2 — a 744-billion-parameter coding model with a one-million-token context window, MIT-licensed open weights arriving within days. The timing was not lost on the developer community. ℹ️ The message landed clearly on Hacker News: as user Reubend put it, they're "grateful to Chinese labs for being open with their work" — especially after "the Fable 5 fiasco." Open weights aren't just a cost play anymore. They're insurance. This guide walks you through actually running GLM-5.2 on your own hardware — the VRAM you need, the quantization that fits, and the exact commands for llama.cpp, Ollama, and LM Studio. No API keys. No cloud dependency. No one can pull the plug. What GLM-5.2 Actually Is GLM-5.2 is the third major iteration in Z.ai's GLM-5 line, purpose-built for agentic coding and long-horizon software engineering . Here is what you are working with: Spec Value Architecture Mixture-of-Experts (MoE) Total Parameters 744 billion Active Parameters ~40 billion per token Context Window 1,000,000 tokens Max Output 131,072 tokens Training Data 28.5 trillion tokens License MIT (open weights) Thinking Modes High and Max The MoE architecture is the key to local viability. Only ~40 billion parameters fire per token — the rest sit idle. That is what makes aggressive quantization work: you are compressing 744B weights, but inference only touches a fraction of them at any given time. GLM-5.2 supports two thinking-effort presets: High and Max. Z.ai recommends Max as the default for coding work — it produces longer reasoning chains before generating output. The model launched on June 13 on Z.ai's C

Originally published at llmkube.com/blog/making-self-hosted-llm-agents-trustworthy . Cross-posted here for the dev.to audience. Running a single local LLM node is a solved problem. You write an InferenceService, the operator schedules it, llama.cpp or MLX serves it, and you get an OpenAI-compatible endpoint. We have been doing that for months. Running a fleet of them is where it stops being easy. My fleet is heterogeneous on purpose: CUDA pods in the cluster, and Apple Silicon Macs sitting off-cluster on the homelab network, each one running two separate agents (one for inference, one for the agentic coding harness). The day I shipped 0.8.4 to that fleet, I learned exactly how it does not scale. I updated each Mac by hand. The control plane had no idea what version any agent was running. And the launchd reload I used to restart an agent was a silent no-op on an already-loaded service, so the old binary kept running while I believed I had updated it. I found that out by hand-inspecting a process tree. Three machines made it annoying. Thirty would make it impossible, and the whole pitch for sovereign, on-prem AI is that you run a lot more than three. So the last stretch of work on LLMKube was not about a faster runtime or a bigger model. It was about making the fleet trustworthy : able to update itself safely, and unable to lie to the control plane about its own state. Here is what that took. Helm and brew for the edge The fix is a new cluster-scoped CRD, AgentRelease , and a self-update path in the agents themselves. You describe the release you want once, the operator rolls it out, and the agents pull and apply it. The design borrows directly from prior art that already solved this for Kubernetes nodes: Rancher's system-upgrade-controller, k0s autopilot's per-platform SHA-256 staging, and Teleport's outbound-only poll model. The properties that make it safe to leave running: Declarative and approved. An AgentRelease names the agent, the version, and the per-platform

You have 50 documents and you're running them through an LLM in a loop. The first one finishes at the 2-second mark. The fiftieth finishes at the 100-second mark — not because it's harder, but because it waited in line behind the other 49. Pool runs all 50 at the same time. The Problem With Loops Every developer who works with LLMs writes this code eventually: import os from yait_aichain.models import Model from yait_aichain.skills import Skill skill = Skill ( model = Model ( " claude-sonnet-4-6 " , api_key = os . getenv ( " ANTHROPIC_API_KEY " )), input = { " messages " : [{ " role " : " user " , " parts " : [ " Summarise in two sentences: \n\n {text} " ]}]}, ) documents = [{ " text " : f " Document { i } content... " } for i in range ( 50 )] results = [] for doc in documents : result = skill . run ( doc ) results . append ( result ) It works. It's readable. And it's painfully slow. Each LLM call takes roughly 2 seconds. Multiply that by 50 documents and you're staring at your terminal for almost two minutes. The calls are completely independent — document 37 doesn't need the result of document 12. Yet document 37 sits idle, waiting its turn. That's a scheduling problem, not a computation problem. I ran into this directly while building a task that pulled N files or links and produced a consolidated report. The sequential version was logically fine but just hemorrhaged time. I needed to fire everything at once without rewriting the Skill logic — no new prompt templates, no restructured code, just a different execution model. That's what Pool is. Pool: Parallel Map for LLM Calls Pool takes one Skill (or Chain) and a list of inputs , then launches all of them concurrently. Think of it as Array.map() where every element runs in parallel against an LLM. import os from yait_aichain.models import Model from yait_aichain.skills import Skill from yait_aichain.pool import Pool , DONE , FAILED skill = Skill ( model = Model ( " claude-sonnet-4-6 " , api_key = os . getenv ( "

You see words. A language model sees tokens — chunks of text, usually a few characters each. Everything starts here. Day 2 of my AIFromZero series. Text gets shattered into tokens "unbelievable" → ["un", "bel", "iev", "able"] (4 tokens, not 1 word, not 12 letters) Before any "thinking", your text is chopped into tokens and each becomes a number the model processes. Why not words or letters? Letters : too fine — the model would relearn spelling everywhere. Whole words : too many — millions, plus every typo and name. Subword tokens : the sweet spot. Common words = 1 token; rare words split into reusable pieces. A fixed ~100k-token vocabulary covers any text. The ~4-chars rule (and why it costs you) In English, ~4 characters ≈ 1 token , or ~0.75 tokens per word. This is how everything is priced and limited: API bills are per token (prompt + reply). A "context window" (how much it can read at once) is measured in tokens — 1,000 tokens ≈ 750 words. Verbose prompts and long chat history burn tokens. Concise prompting is a real cost lever. The strawberry problem The model never sees s-t-r-a-w-b-e-r-r-y. It sees a token like straw + berry . The individual letters are buried inside tokens, so counting characters is genuinely hard for it. It's not dumb — it just doesn't read letters. Tokens are step 1 of everything Tokenize → turn each token into a vector (embeddings, next) → run through the transformer → predict the next token. Every LLM starts exactly here. 🔤 Type anything and watch it tokenize live: https://dev48v.infy.uk/ai/days/day2-tokens.html Day 2 of AIFromZero.

The earlier posts in this series were about what the gateway lets you call (cache-aware spawning across five providers, the Codex review gate, the CLI-versus-API argument) and the one before this was about the parts that do not show up as a tool, the upstream-tracking and the website that became the project's front door. This one is about a different kind of front door: the gateway can now listen over HTTP, behind real authentication, and serve more than one caller without those callers being able to read each other's work. That sounds like a small toggle. It is not. Moving an MCP server off a local pipe and onto a network port changes the trust boundary completely, and 2.9.0 is the release where I sat down and remediated all seventeen findings from a multi-LLM red-team of exactly that surface before telling anyone the remote path was ready. Short version: llm-cli-gateway is one Model Context Protocol server that wraps five vendor CLIs (Claude Code 2.1.177, Codex 0.139.0, Google's Antigravity agy 1.0.8, Grok 0.2.51, and Mistral Vibe 2.14.1) behind a single, uniform tool surface, so one orchestrating agent can fan a task out to several models, collect independent opinions, run a red-team or a consensus check, and keep durable session and job state across all of it. Until recently that only made sense on localhost over stdio. As of 2.9.0 the same server runs over HTTP with a static bearer token or a built-in OAuth 2.0 authorisation server (PKCE on by default, an opt-in human-consent gate, and a trusted-principal-header seam for when you front it with your own identity-aware proxy), every session and job and stored request is stamped with an owner principal and access is enforced per principal, remote provider calls are refused unless a workspace is registered, and the whole thing fails closed rather than open when the configuration is dangerous. Long version is below, same shape as always: what it enables, what the remote options actually are, a stack of worked scenar

Extends an earlier model-selection benchmark to three model families (Japanese / Western / Chinese) on a Japanese RAG task. Repo + raw results: https://github.com/elvisyao007/eval-driven-llm/tree/main/reports/model-selection-v2 An earlier post benchmarked local models for a Japanese RAG task and settled on selecting by constraint rather than raw capability. This post widens the field to three families — Japanese-tuned, Western open, and Chinese — and the result forces a distinction that matters more than any single score: model capability and deployment eligibility are two different questions, and conflating them is how people get model selection wrong. Same Japanese RAG task, same judge protocol, same discriminating golden set (oracle 87.5%, only 11% of questions answered by all models — it actually separates the field). hit@5, 8B class unless noted: Model Family hit@5 Swallow-8B Japanese-tuned ~0.53 Nemotron-9B-JP Japanese-tuned ~0.62 ELYZA-JP-8B Japanese-tuned ~0.40 deepseek-r1-8b Chinese ~0.51 Llama-3.1-8B Western ~0.22 Mistral-7B Western ~0.18 gemma4-31b Western (31B) ~0.62 Three things fall out of this, and they don't all point the same direction. 1. At 8B, Japanese fine-tuning is decisive — and generic Western models just aren't competitive The Western 8B models cratered: Llama-3.1-8B at 0.22, Mistral-7B at 0.18, against a Japanese-tuned average around 0.52. That's not a small gap; it's the difference between usable and not. This answers a question people sometimes ask skeptically — why do Japanese-specific models exist when Llama is right there? At the 8B scale, on a Japanese retrieval-grounded task, a generic Western model without Japanese fine-tuning is not in the running. The Japanese tuning is doing decisive work. One honest qualifier on the table: gemma4-31b (0.62) is the one Western model that holds up — but it's 31B, not 8B. It earns its score with 4× the parameters, not with Japanese optimization. So read the table in two tiers: within the 8B class,

Most RAG demos answer "what's the right chunk?" Very few can answer the two questions a regulator or an auditor will actually ask: Replay this decision — show me the exact, complete record of how this answer was produced. Reconstruct the past — what did your system know at the moment it answered, not what it knows now? I got tired of hand-waving at both, so I shipped two pre-registered, deterministic benchmarks alongside JAMES , my local-first, audit-native Graph-RAG. Pre-registered means the metrics, scenarios, and decision rules were locked before the numbers came in — no post-hoc story-fitting. RAB — Replayable-Audit Benchmark RAB measures whether your audit trail is good enough to replay a decision, with three deterministic metrics: Metric What it checks EU AI Act AC — Audit Completeness Is every decision-relevant event logged? Art. 10 RF — Replay Fidelity Can you re-derive the answer from the log alone? Art. 12 PC — Provenance Coverage Does every claim trace to a source? Art. 19 The three metrics map verbatim to EU AI Act Articles 10, 12, and 19 — record-keeping obligations that apply from 2026-08-02 (per Article 113). Scenario S1 result: AC RF PC JAMES 1.000 1.000 1.000 Baseline-0 0.275 0.000 0.000 (vanilla default-logging) The gap is the whole point. "We have logs" (AC 0.275) is not the same as "we can replay the decision" (RF 0). Default application logging gets you a partial event trail and zero replay/provenance — which is exactly the failure mode an Article 12 audit would surface. LRB — Lifecycle Retrieval Benchmark RAG facts go stale. A policy is superseded, a price changes, a spec is revised. LRB asks: when you query as of a point in time, do you retrieve the fact that was valid then , or whatever overwrote it? Three systems compared: V — Vanilla : no time handling. N — Naive-supersede : newest fact wins. J — JAMES : validity-window retrieval ( reconstruct_graph_at(t) ). The R@1 ordering V < N < J holds across 4 model families × 4 scale points (a 12.5×

The story of AI for the last three years has been written in megawatts. Nvidia GPUs stacked in desert data centers . Models with trillion-parameter counts. APIs that pipe your prompts, photos, and personal data to the cloud, burn a forest of electricity to process them, and return an answer 800ms later. If you're building with AI in 2026, the default assumption is that intelligence lives somewhere else. Your device is just a glass terminal. Apple has been telling a different story. No press tour. No "AGI in your pocket" hype cycles. Instead, a decade of silicon releases where the Neural Engine number (FLOPS) quietly doubled, then doubled again. Core ML updates that casually added transformer support. Here is my thesis : Apple’s on-device AI strategy is a privacy-first, performance-oriented architectural break from cloud-centric AI. By co-designing silicon, models, and APIs to run locally, Apple is unlocking a new class of local-first applications where user data never leaves the device, latency is measured in milliseconds, and features work in airplane mode. This doesn’t kill cloud AI. But it forces every developer to answer a new question: what part of your product must be in the cloud, and what gets better when it stays in the user’s pocket? This post is a technical teardown of that shift. I’ll cover the hardware realities of the Neural Engine and unified memory, the brutal constraints of fitting LLMs on device, what Core ML actually gives developers in 2026, and where this architecture creates new product opportunities that cloud-first can’t touch. I’ll also be blunt about the limits. On-device AI won't replace GPT-5 training clusters. But it might replace 80% of the API calls you make to them. At this moment, cloud AI gets all the headlines, but the real transformation may already be running 24/7 in your pocket, without ever touching the internet. Why the On-Device Push? Apple's AI strategy looks slow only if you measure it in keynote superlatives. Measure it in

Introduction Zhipu AI (THUDM) has officially released GLM 5.2 , the latest iteration of its flagship open-weights model family. Announced today by Jie Tang on Twitter, the release is already making waves on Hacker News — racking up 269 points and 146 comments within hours. For developers who have been watching the open-weight LLM race, this is a significant moment. What's New in GLM 5.2 GLM 5.2 builds on the GLM-4 series that put Zhipu on the global map. The release focuses on three areas that matter most to production teams: Stronger reasoning and coding : Improved performance on multi-step reasoning benchmarks and competitive code generation against closed-source models like GPT-5 and Claude 4.5. Better multilingual behavior : GLM has always been strong in Chinese; 5.2 pushes English-quality code reasoning and longer-context retrieval closer to frontier levels. Longer context window : Reports point to a 200K+ token context with reduced degradation on long-document tasks — useful for codebase-level analysis. Weights, inference code, and a technical report have landed on Hugging Face under the THUDM organization, with an OpenAI-compatible API endpoint exposed by Zhipu's own platform. Why It Matters The open-weights race has consolidated around a handful of serious contenders — Llama, Qwen, DeepSeek, Mistral, and now GLM. Zhipu's positioning is unique: a Chinese lab that consistently weights-and-releases frontier-class models while still maintaining a hosted commercial API. For developers, that translates to real options: You can self-host on a single H200 or a pair of RTX 5090s and skip per-token API costs entirely. You can route between self-hosted GLM 5.2 and a hosted Anthropic/OpenAI endpoint depending on cost, latency, and capability. You get an OpenAI-compatible endpoint, so dropping GLM into an existing stack is a config change, not a rewrite. The Bigger Picture GLM 5.2 lands on the same week that U.S. regulators have reportedly cracked down on Anthropic model

Six months ago, I could tell you which model to use for almost any job, and I would have said it with confidence. Today I hedge, and so does almost everyone I talk to who builds with these tools. The reason is simple. The ground keeps moving under us. Models get smarter on a schedule no one can forecast, and they get cheaper to run on a second schedule that is just as hard to predict. Both curves are bending at once, and they point in directions that change how I build and how I think you should build too. I spend my days crafting development, content and productivity workflows that lean on these models. I wire up agents, route tasks, and watch the bills. So this is not a far-off observation for me. It is the thing I am living with week to week, and it has forced me to rethink habits I held for years. This is not the usual story about a single breakthrough. It is four shifts happening together. Frontier performance is climbing past what most of us guessed was possible this year. Small models are getting good enough to run on a phone or a thirty-five dollar computer. The smart move has stopped being "pick the best model" and started being "build a system that picks for you." And a coding startup with a rocket company behind it is showing what happens when product, data, and compute sit under one roof. Let me take these one at a time. Then I want to show you what they add up to, because the sum is bigger than the parts. Performance Is Outrunning the Forecasts Start at the top of the market, where the most capable models live. Anthropic now ships a tier above its Opus line. The Fable and Mythos family is a class of model built for problems that smaller systems still fumble: long chains of reasoning, deep code work, research that needs to hold many threads at once. Claude Fable 5 carries extra safety work so it can go out to the public. A more powerful sibling, used inside a small set of trusted partners, stays behind tighter controls. The names are not the point. The p

CORE had telemetry. That was the comforting part. Every LLM exchange was being logged. Prompt tokens. Completion tokens. Duration. Cognitive role. Model snapshot. Timestamp. Privacy level. Enough information to reconstruct what the system had asked, which model had answered, and how the autonomous loop had used the result. Then I asked the obvious question: What did the last month of LLM work cost? The database had no answer. Not a bad answer. Not an approximate answer. No answer. The cost_estimate column existed. It was even part of the log model. But across 35,669 recorded LLM calls, it was populated exactly zero times. Every row was NULL. That is the kind of bug that looks small until you understand what kind of system CORE is trying to become. CORE is not just a wrapper around LLM calls. It is a governance runtime for AI-assisted software development. The point is not that an AI writes code. The point is that every AI-produced change must be traceable, authorized, constrained, audited, and defensible. So when cost attribution was missing, this was not just a FinOps bug. It was a governance blind spot. The System Could Explain the Work, But Not the Bill The strange thing was that most of the telemetry was already there. CORE knew which cognitive role made the call. It knew whether the call came from an architect, coder, reviewer, coherence analyst, or some other internal role. It knew which model handled the request. It knew the token counts. It knew when the call happened. That meant I could ask questions like: Which cognitive roles are consuming the most tokens? Which models are being used by which part of the system? Which workflows are driving LLM activity? How much autonomous reasoning happened during a given period? But I could not ask: Which cognitive role costs the most? Did routing this role to a stronger model actually change the cost profile? Did a model swap increase operational cost? Is local inference replacing paid inference in the places where it

LLM API Reliability: The Reality Nobody Talks About If you have run more than a few thousand LLM calls in production, you have seen the pattern: things work perfectly in development, then fall apart under load. The Numbers Failure Type Rate Root Cause Timeout 2-5 percent Network congestion, provider throttling Rate Limit (429) 1-3 percent Burst traffic patterns Empty Response 0.5-2 percent Content filtering, model degradation Schema Violation 1-4 percent Model behavior drift 5xx Server Error 0.5-1 percent Provider-side outages Total: 5-15 percent of calls fail on first attempt. Why Retry-Only Is Not Enough Most teams implement exponential backoff and call it done. But retry alone does not help when: The provider is genuinely down (retrying into a black hole) The model has degraded silently (retrying returns the same bad output) You are being rate limited (retrying makes it worse) Self-Healing: A Better Approach Instead of naive retries, a self-healing approach: Diagnoses the failure type (~19 microseconds) Escalates through layers: retry, degrade, failover, learned rule Validates output quality across multiple dimensions Learns from each failure for next time Key Takeaways 5-15 percent of production LLM calls fail on first attempt Retry-only strategies fail when providers are degraded Self-healing with diagnosis and failover recovers 84.1 percent of faults Multi-provider routing eliminates single points of failure Try It https://github.com/hhhfs9s7y9-code/neuralbridge-sdk NeuralBridge is Apache 2.0 open source.

Show HN: NeuralBridge — We Built a Self-Healing SDK for LLM-Powered Agents After months of production experience running LLM calls at scale, we realized something uncomfortable: every AI agent eventually crashes . Not because the code is wrong, but because LLM APIs fail in ways you can't predict. Timeouts. Rate limits. Empty responses. Schema violations. Drift. These aren't edge cases — they're the norm. So we built NeuralBridge: an embedded SDK that makes LLM calls self-healing. The Problem Try running 100,000 LLM calls through any single provider. You'll see: 2-5% failure rate from timeouts and 5xx errors Rate limits that cascade through your pipeline Schema violations when models change behavior Provider-specific quirks that require custom error handling 30-200ms of unnecessary latency from gateway proxies Most teams solve this by building their own retry logic, circuit breakers, and fallback chains. It works — until it doesn't. Because the next failure is always the one you didn't anticipate. Our Approach: Embedded Self-Healing Instead of a gateway (which adds latency and infrastructure), we embedded the reliability logic directly into the SDK: from neuralbridge import SelfHealingEngine engine = SelfHealingEngine () result = engine . call ( " Write a Python function for binary search " ) if result . recovered : print ( f " Fault: { result . diagnosis } " ) print ( f " Recovery: { result . recovery_action } " ) When a call fails, the engine: Diagnoses the fault type in ~19us (P50) Escalates through 4 layers: retry -> degrade -> failover -> learned rule Validates the output across 5 dimensions Learns from the experience for next time Production Results Metric Value Auto-recovery rate 84.1% of faults Fault patterns recognized 280+ Recovery strategies 30+ Learned rules (flywheel) 88+ Diagnosis latency 19us P50 Install size 375 KB Why Open Source? We went Apache 2.0 because reliability infrastructure should be a commodity. The SDK is free and open. Pro features (ente

What is NeuralBridge? NeuralBridge is an embedded SDK (not a gateway) that makes your AI agents resilient against LLM failures. It runs inside your Python process — zero infrastructure, zero HTTP proxy, one dependency. pip install neuralbridge-sdk Your First Call import neuralbridge as nb result = nb . run ( " Explain quantum computing in one sentence " ) print ( result . text ) That's it. NeuralBridge auto-discovers your API keys from environment variables and handles multi-provider routing, self-healing, and drift detection automatically. What Makes It Different? Self-Healing Engine — When an LLM call fails (timeout, rate limit, bad response), NeuralBridge doesn't just retry. It diagnoses the fault type, degrades gracefully, fails over to another provider, and learns from the experience. from neuralbridge import SelfHealingEngine engine = SelfHealingEngine () result = engine . call ( " Write a Python function for binary search " ) print ( result . flight ) # Shows diagnosis, recovery action, latency print ( result . recovered ) # True if self-healing was activated 84.1% of production faults are auto-recovered. 19us diagnosis time P50. Why Not a Gateway? Every gateway (LiteLLM, etc.) adds 30-200ms of network latency. NeuralBridge runs in-process, adding zero additional latency. Approach Latency Dependencies Deployment Gateway (LiteLLM) +30-200ms Docker + PostgreSQL Ops team SDK (NeuralBridge) +0ms 1 (httpx) pip install Key Features 4-Layer Self-Healing : L1 retry -> L2 degrade -> L3 failover -> L4 flywheel 5-Dimension Validation : JSON Schema, semantic, entity, taboo, composite Multi-Provider Routing : DeepSeek, OpenAI, Anthropic, and 12+ more Drift Detection : Catch model regressions before users do Carbon Tracking : Per-provider carbon footprint per call Open Core : Apache 2.0 license, 375 KB install size See It in Action import neuralbridge as nb result = nb . run ( " Hello " , providers = [ " openai " , " deepseek " ]) print ( f " Used provider: { result . prov