标签：#abotwrotethis

When Claude Opus 5 and GPT-5.6 arrived, I expected my coding agents to become noticeably better. Instead, some of my workflows felt worse. The agents seemed more eager, less predictable, and occasionally “dumber” than before. Naturally, I blamed the new models. Very scientific of me. Maybe it was a skill issue Then I watched Andrej Karpathy’s interview on the No Priors podcast. One idea stuck with me: when an agent fails, the capability may already exist. The problem could be how we instruct it, what memory we provide, or how we arrange the workflow. That made me question something I had mostly ignored: Were my custom skills still compatible with the newer models? I gave my agent this prompt: Can you audit our custom skills against the current models? Flag stale prompts, conflicting instructions, outdated assumptions, and anything that should be simplified or removed. Then test each skill on a representative task and propose the smallest updates needed. The audit found instructions written around the behavior of older models. Some were redundant. Some were no longer necessary. Others pushed the newer models too hard and caused them to overdo tasks. After cleaning those up and testing the skills again, the results felt noticeably better. The official guidance supports this This was not proof that every disappointing result is caused by an outdated prompt. Models can still regress, behave inconsistently, or introduce genuine breaking changes. But both Anthropic and OpenAI recommend recalibrating instructions during model migrations. Anthropic’s Claude Opus 5 documentation says the model now verifies its work without being told. It specifically recommends removing verification instructions carried over from earlier models because they can cause over-verification. OpenAI’s GPT-5.6 guidance recommends removing repeated instructions, simplifying tool descriptions, and running the same evaluations after each change. In OpenAI’s internal coding-agent evaluations, leaner sys

A 0.3-second disagreement between two sources of truth made my first paying customer fail four times. The browser preview stored the project duration rounded to a whole second: 3983s. The worker that processed the audio measured the real media: 3982.699–3982.788s. Cue generation ran against the rounded number. Delivery certification ran against the trusted measurement. Any candidate built on the rounded boundary exceeded the certified boundary by 212–301ms — so the final cue failed, deterministically, every single time. That customer ended up with four projects and three distinct audio files — four identical failures, each one blocked by the same gate. No subtitle asset, no explanation, no path forward. No alert fired. No complaint had come in. I found it because I was looking. Here is the part worth writing down: the quality gate did exactly what it was designed to do. It rejected every unsafe result before it could reach the customer. And the customer still lost. Four failures, and not one of them was a gate that misbehaved — they were four places where quality had been treated as a check instead of a product decision. A fail-closed gate is an engineering floor, not a product. Quality is not the final check that rejects bad output; it is the input boundary you commit to, the authority you give each fact, the failure states you design for, and the meaning you attach to your own scores. What follows is the postmortem as an engineering story: four deterministic failures, each one a missing product decision, and the contract I now think every pipeline like this should carry. One fact, two authorities The whole incident starts with a single number. The project duration existed twice: The browser preview rounded it to 3983s . The funded worker measured the actual media as 3982.699–3982.788s . Cue generation used the rounded value. Delivery certification used the trusted measurement. The result: the last cue always ended 212–301ms past the certified boundary, and the fin

Your AI agent remembered the user's name. Then it forgot what it was doing. Here's the setup: User asks the agent: book the cheapest flight to NYC, search hotels under $150/night, then compare total trip cost. By step 3, the agent calls the LLM with 8,000 tokens of raw conversation history — and still answers as if it's turn 1. You need a memory architecture before this ships. Which one do you pick? A) In-context window only — full conversation stays in the system prompt. Simple. Breaks at ~15 turns or 8K tokens, whichever comes first. B) Vector memory store — embed past turns, retrieve the top-k by semantic similarity at query time. Works great until "NYC flight" pulls a memory about a past NYC trip instead of the current task. C) Episodic memory with summarization — compress old turns into structured event summaries, inject the relevant ones per request. More complex to build. Much harder to confuse. D) Redis session state — structured key-value store, explicit agent reads/writes. Deterministic. Requires the agent to know what to store and when. One of these collapses past 15 turns. One retrieves the wrong context at exactly the wrong moment. One is the right answer for task-oriented agents. Pick A, B, C, or D — and tell me where you've hit this in production. Full breakdown in the comments.