Disclosure: I maintain Lynkr , an open-source router whose design decisions this post explains. The failure modes described are patterns widely reported across router issue trackers and local-LLM forums — the examples are representative reconstructions, not captured transcripts. The problem is real either way; ask anyone who's routed a coding agent to a 7B model. Everyone who gets their first LLM router working does the same thing within the hour: point the expensive coding agent at a free local model and watch the bill drop to zero. Then the agent tries to edit a file. The graveyard of downgraded sessions If you browse the issue tracker of any Claude Code router — or r/LocalLLaMA on any given week — you'll find the same story in a hundred variations. The routing works perfectly. The session dies anyway. The killers, in rough order of frequency: 1. Malformed tool arguments. The agent decides to call Edit , and the model produces arguments that are almost JSON: { "file_path" : "src/auth.js" , "old_string" : "if (token) {" , "new_string" : "if (token && !expired) {" One missing brace. The harness rejects the call, the model retries, produces a different malformation, and you're three turns deep into fixing nothing. Frontier models emit structurally valid tool calls with boring reliability; sub-10B models do it most of the time — and "most of the time," at 30 tool calls per session, means every session breaks. 2. Stale string matching. Edit -style tools require the old_string to match the file exactly. Small models paraphrase from memory instead of quoting — they'll "remember" the line as if (token) { when the file says if (accessToken) { . The edit fails, the model re-reads the file, burns 2,000 tokens, tries again with a different paraphrase. This is the single most reported failure, because it looks like the router's fault and is actually a capability cliff. 3. Hallucinated context. Ask a small model to run tests and it may confidently call Bash with npm test -- --g

A lot of AI memory systems start with the same temptation: "Just save the useful thing." That sounds harmless until the knowledge base becomes a junk drawer. Half the notes are too specific, a few are duplicates, some are obsolete, and nobody knows which ones the agent should trust. In ai-assistant-dot-files , the memory system is deliberately slower. It uses a promotion lifecycle: Capture -> Candidate -> Audit -> Approve -> Index -> Retrieve -> Expire That lifecycle is documented in docs/runbooks/memory-engineering.md , and the important word is not "capture." It is "candidate." Nothing writes directly to memory The framework has a durable memory layer: Knowledge Items in shared/knowledge/ , ADRs in docs/adrs/ , the domain dictionary, team topology, a feature archive, and a registry at shared/memory-registry.json . But a lesson from a delivery does not jump straight into shared/knowledge/ . It first becomes a Candidate Record. That record has required fields: Source Type Evidence Tags Expiration condition Then memory-engineer audits it: Is it reusable? Is it already covered? Is it too speculative? Does it belong as a Knowledge Item, or should it become a rule change, prompt edit, or ADR instead? Only after that does a human approve the destination. The design is intentionally similar to code review. Durable memory changes future behavior, so they deserve a paper trail. Rejection is a feature One of my favorite parts of the memory runbook is that it has explicit rejection rules. Do not promote a memory when it is: a one-off already covered too speculative That makes "zero candidates promoted this cycle" a healthy result, not a failure. This is where memory engineering starts to look less like note-taking and more like gardening. The point is not to preserve every leaf. The point is to keep the soil useful. Expiration matters The lifecycle also includes expiration. A Knowledge Item can become stale when the underlying code, agent, or pattern changes. It can be supers

React Doctor es un linter de cero instalación que escanea un codebase de React buscando problemas de correctitud, seguridad, accesibilidad, rendimiento y mantenibilidad, luego los rankea y te entrega una lista de arreglos con forma de agente. Este post es un reporte de campo: lo corrí sobre una SPA real de React 18 + Vite + TypeScript (~50 rutas, TanStack Query, react-hook-form) y separé lo que de verdad me atrapó. El resultado honesto: 1,249 hallazgos, cinco que valía la pena arreglar, incluyendo dos bugs de seguridad reales. Los otros 1,244 fueron una mezcla de ruido, decisiones de criterio, y falsos positivos. TL;DR Categoría Reportados Vale la pena actuar Por qué la brecha Seguridad 18 warnings 2 13 eran sinks saneados en el servidor (falsos positivos); 1 mina de código muerto + 1 XSS real fueron el oro Bugs 24 errores, 487 warnings 2 1 fuga de timer, 1 key/spread; ~27 exhaustive-deps piden criterio humano Accesibilidad 434 warnings, 1 error 1 el 1 error ( aria-selected faltante) era real; los 434 son ruido de <Label> / <button type> Rendimiento 110 warnings 0 nada caliente; candidatos pero sin impacto medido Mantenibilidad 176 warnings 0 opiniones de estilo tipo "componente grande" Total 1,249 5 señal-a-ruido ≈ 1 en 250 El puntaje que imprimió: 39 / 100 . Dos cosas que ese encuadre esconde: los cinco que sacó a la superficie eran de alto valor (una fuga de token en localStorage y un sink de XSS almacenado), y el conteo no es determinista : una segunda corrida del mismo commit reportó 1,254, no 1,249. El veredicto en una línea: excelente generador de hipótesis, pésima compuerta. Úsalo para encontrar candidatos, verifica cada uno contra el código, y nunca conectes el conteo al CI. Qué es React Doctor Un solo binario que corres por npx / pnpm dlx , sin agregar dependencia a tu proyecto, sin archivo de configuración obligatorio. Parsea tu src/ , hace match contra un conjunto de reglas (sus familias: Seguridad, Bugs, Accesibilidad, Rendimiento, Mantenibilidad), e im

We deployed a coding agent that hit 94% on the industry benchmark. It failed in production on the first real edge case because the benchmark measured single-turn success and our actual work was multi-turn refinement. The model could not update its beliefs correctly when new evidence arrived, something no single-turn eval would catch. This is not a hypothetical. I have watched agents shine in demo and disintegrate on the messy input that production actually serves. The gap between what we measure and what ships is real, and it is where reliability lives or dies. The benchmark misses the point FutureBench evaluates agents by asking them to predict events that occurred after their training cutoff. This removes the possibility of correct answers coming from memorized training data rather than genuine reasoning. The design matters because it tests whether an agent can reason, not whether it can recall. BayesBench showed that standard LLM evaluations score only final-turn answers in single-turn format, leaving multi-turn belief updating entirely unexamined. Across seven models, scaling improves latent inference and evidence accumulation but LLMs do not match rational Bayesian updating. In production, your agent runs many turns. The benchmark that stops at turn one is not measuring the thing that actually breaks. KINA identified three systematic flaws in knowledge benchmarks: scaling-driven designs that ignore disciplinary representativeness, flat-payment annotation that permits lazy consensus among annotators, and unaudited ranking instability under bounded test budgets. The top model reached 53.17% on an 899-item benchmark across 261 disciplines. That is not saturation. That is headroom. The demo lied I worked with a team that deployed an agentic document processing system. The demo on ten handpicked cases was flawless. The first week of production, it hit an input format the training data never saw, and the system failed silently. No error was raised. The output looked

A new generation of voice models for natural human-AI interaction, now powering ChatGPT Voice.

I'll preface to say I'm not at all a programmer, so pardon me for saying anything stupid The increasing amount of slop is no secret, and but software as a whole, be it plugins or apps or even whole operating systems, has become increasing plagued by a set of kinda-unrelated problems that feel like they fall under the same umbrella. I'll keep it short, so practical issues of efficiency, privacy, security, and long-term support, and then ethical issues of being human-made. I personally believe that we're in need of novel, widespread terms to be able to describe the kind of software that in principle does not have certain traits. I attached the link because I believe the term "open source" is analagous to what I'm talking about here. Everybody has an idea of what it means and entails, but it also doesn't mean that everything has to strictly follow it to a T, because it's just a word at the end of the day. It's a pledge to the user to abide by principles of being "Open Source," and I'm saying that we need new words that similarly serve as principles such as that of, optimization, long-term support, being human-made, etc. (It's kind of like how we have terms like organic, free-run eggs, free range eggs, and of course you would end up with having a bunch of terms that dont mean anything at all, but i'm confident that a few will emerge as "standards" in the same way the term open source is.) This could go hand-in-hand with maybe making it common practice to at least disclose the development process when publishing software. In (science) research, we're beginning to see the implementation of "AI Disclosures" to detail all such usage, to be published with the paper. surely practices like these should be encouraged in software development as well. I'm sure there are (smaller) discussions on what exactly should constitute as "best practices" to the people that care about such things, but I've yet to see mainstream terms that go beyond only vaguely describing what we dont like

I spend a lot of time preaching architecture and constraints, so it is always nice when a side project gives me receipts. Adding this new feature to DumbQuestion.ai was a good reminder that a well-structured first version lets you spend your next iteration on value, not repair. Below, you will find a few relatively simple challenges and how thoughtful, upfront design made the changes effortless. To vibe or not to vibe ... Many developers jump right in and just rip out an app, ship fast, let the coding agent sort it out, come back and deal with it later. To be fair, that absolutely can get you to first release faster. But even on a solo project, a little proper SDLC discipline pays back later when you want to extend the product without turning every feature into a rescue mission, which is a theme that already runs through how I have been building DumbQuestion.ai. Extend this to the enterprise and you turn a little upfront effort into potential huge savings on token spend Roasting starup pitches (for sport) ... The core idea for Startup Roast was simple enough: take a startup pitch, roast it, and add a reality-check section so the output is not just mockery for mockery’s sake. To illustrate (and avoid just vaguely describing the feature) I picked a random but highly upvoted pitch from Product Hunt: Vida . Vida, which pitches itself as an “AI clone” that learns how you work, remembers what matters, and becomes a “second you,” with early use cases like Reply Rescue, Prompt Rescue, Resume Rescue, Workspace Cleanup, and Daily Wrap. This is a pretty common target use case of agentic AI making it a solid candidate. If you want to skip ahead, here's an example roast for Vida. Combining a preliminary web "market search" into the content yielded a result that was not just sarcastic, but informed. The roast hit the obvious AI-clone positioning, questioned whether the product was really a clone versus a macro suite, and then turned the market context into a sharper Reality Check

Amid public backlash over its smart glasses, Meta announced that it will be updating its glasses with a new feature that will disable the camera when it detects that someone has tampered with or destroyed the glasses' privacy LED light. The update is meant to address modders who have taken actions such as physically drilling […]

Chat with Notion Agents anytime Discussion | Link

Everything worked. I'd wired up useOptimistic on a task list, the checkbox flipped the instant you...

Open source agents that run 100% locally on your Mac Discussion | Link