标签：#Engineering

"A HashMap helps you find what you already know. A Heap helps you decide what should happen next." In the previous article, we discovered that not every software problem is about finding a specific object. Sometimes, the system already knows exactly what it's looking for. Find User ID = 1024 ↓ Return User Other times, the system doesn't know the answer in advance. Instead, it has to repeatedly answer questions like: Which task should run next? Which driver should be assigned? Which customer should be served first? Which alert is the most critical? These are fundamentally different problems. Instead of retrieving an object, the system is making a decision. This is where a Heap comes in. A Heap Is Built for Decisions, Not Searches Imagine you're managing a hospital emergency room. Patients keep arriving throughout the day. Patient A Minor Injury Patient B Heart Attack Patient C Broken Arm Patient D High Fever Should doctors treat patients in the order they arrived? Probably not. Instead, they ask one question. Who needs treatment first? Notice something important. The hospital isn't searching for a particular patient. It's choosing the highest-priority patient. A Heap is designed for exactly this kind of problem. A Different Way of Thinking When beginners hear "data structure," they often think about storing data. Experienced engineers think differently. They ask: "What operation does my system perform repeatedly?" If the answer is: Find User Find Order Find Product that's a lookup problem. But if the answer is: Choose Highest Priority Choose Nearest Driver Choose Earliest Deadline that's a decision problem. A Heap is optimized for continuous decision-making. What Exactly Is a Heap? A Heap is a data structure that keeps the most important element immediately available. Depending on the system, "most important" can mean different things. For example: Highest priority Lowest cost Earliest deadline Highest score Closest driver Most urgent ticket The Heap doesn't decide w

"Finding something quickly and finding the best thing quickly are two completely different engineering problems." So far in this series, we've explored one of the most common behaviours in software systems: Fast lookup. Whenever a system already knows what it's looking for—a User ID, Product ID, Order ID or Session ID—a HashMap becomes an excellent choice. But not every software problem works this way. Imagine you're building a ride-sharing application. A rider requests a cab. The system doesn't already know which driver to assign. Instead, it must answer a different question: "Out of all available drivers, who is the best choice?" Now consider a task scheduler. Hundreds of jobs are waiting to run. The scheduler doesn't ask: "Find Job #123." It asks: "Which job should run next?" Or imagine a gaming platform. Thousands of players are competing. Nobody asks: "Find Player ID 1057." Instead, users ask: "Who are the top 10 players?" These problems are fundamentally different from fast lookup. They're not about finding a specific object . They're about finding the best object according to some priority. This shift in thinking introduces another important design behaviour. Fast Lookup vs Best Selection Let's compare two different requirements. Requirement 1 Customer ID = 1052 ↓ Retrieve Customer The system already knows exactly what it needs. The challenge is retrieving it efficiently. Requirement 2 Available Drivers ↓ Find Nearest Driver ↓ Assign Ride The system doesn't know the answer yet. It must compare multiple candidates before making a decision. These two behaviours may look similar. In reality, they solve completely different engineering problems. Every Software System Doesn't Search the Same Way Consider these questions. Find Order #50231 versus Find the highest priority order. Or: Retrieve Product ID = P1042 versus Recommend the most popular product. Or: Find Employee ID = 2107 versus Find the employee with the highest sales this month. The first question always

Most agent tooling is still built around a single conversation: one agent, one task, one terminal, one stream to babysit. Fine for small tasks, bad for real engineering work. Herdr is interesting because it treats that as the default shape of the work. The simplest way to describe it is: Herdr is tmux for coding agents. More precisely, it is an agent multiplexer that runs inside your existing terminal. It gives each agent a real PTY, keeps processes alive where the work is happening, shows agent state, and exposes a CLI plus a local socket API. That distinction matters. Herdr is not another desktop agent app. It is a binary you run where the code and terminals live: a server, a Mac Mini, a VM, a dev machine under your desk. Close the laptop, detach, ssh back later, reattach, even from a phone. The work did not die because your terminal window did. The throughput problem Coding agents changed the cost of starting work. I can ask one agent to explore a bug, another to write a failing test, another to draft a migration plan. The bottleneck is supervision. The problem is that normal terminals do not understand supervision. tmux and Zellij give you persistence and panes, but they do not know whether an agent is blocked, working, done, idle, or just sitting there after printing a question three screens ago. Desktop apps often understand the agent state better, but then the workflow is stuck to the machine with the GUI. Worktree orchestrators can coordinate parallel tasks, but they usually want to own the workflow. Herdr sits in a useful middle: terminal model, plus agent awareness. The performance multiplier is not magic. It comes from four practical properties: Multiple agents run in real PTYs, each with its own shell, logs, prompts, and process state. Herdr rolls up semantic state, so you can see which agents are blocked, working, done, or idle. The server owns the panes, so sessions survive client detach, laptop sleep, and terminal death. The CLI and socket API let scr

How AI Is Transforming Software Development Workflows in 2026 By 2026, AI has moved far beyond autocomplete and boilerplate generation. It has become an integral, intelligent partner in the entire software development lifecycle. From writing initial architecture to diagnosing production incidents, AI agents are embedded into the fabric of modern engineering workflows. This transformation is not just about speed—it's a fundamental shift in the way developers think, collaborate, and deliver software. The Rise of AI-Native Development Environments The days of classic IDEs with a chat sidebar bolted on are behind us. In 2026, AI-native development environments are the norm. These IDEs are built around context-aware AI models that understand not just the syntax but the semantic intent of the codebase. Tools like Cursor and Windsurf have evolved into full-blown autonomous agents that can navigate large codebases, propose cross-file refactors, and even execute multi-step changes with minimal supervision. Consider a common task: adding a new payment gateway. In a traditional workflow, a developer would manually trace API routes, update database schemas, and write integration tests. In 2026, the developer simply describes the requirement in natural language. The AI agent explores the existing adapter patterns, creates the new integration, updates configuration files, and runs the test suite. The developer reviews the diff, tweaks edge cases, and signs off. This paradigm shift has accelerated feature delivery by an order of magnitude. Intelligent Automated Testing and Debugging Testing has always been a critical yet time-consuming part of development. AI in 2026 has revolutionised this domain. Instead of writing every test case manually, developers use AI to generate exhaustive test suites that cover edge cases, security vulnerabilities, and performance bottlenecks. The AI analyses the code's control flow, historical bug data, and production logs to generate tests that would

The Core Argument : AI agent reliability isn't achieved by "making the agent smarter" — it's achieved by the simple engineering principle of separating validation from generation . Quality isn't accidental. It's designed. What You'll Learn : Maker/Checker separation, 6 termination conditions, and an automated feedback loop — all with runnable code. 0. Prerequisites Python ≥ 3.10 OpenAI API Key (or compatible interface) pip install openai>=1.0.0 (Optional) pip install anthropic>=0.30.0 if using Claude as Checker 1. The Pain: Why "Agent Checks Itself" Is a Trap 1.1 The Copying of Cognitive Bias A team built a data-analysis agent. It pulled sales data from a database and generated business reports. The team added a "self-review" step: after generating, the agent told itself "please check if the data you just output is accurate." Result? The agent always replied "data is accurate." Even when the team deliberately injected obvious errors (e.g., monthly sales of -50M RMB), the agent confidently said everything was fine. This isn't the model being "disobedient." It's a more fundamental issue: when the generator and checker are the same entity, the check is just a restatement of the generation process — not real validation. The checker carries the exact same cognitive bias, knowledge boundaries, and reasoning path as the generator. 1.2 The Amplifier Effect of Confirmation Bias Self-checking also triggers a subtler problem: confirmation bias amplification. The model builds a "belief state" during generation; when re-examining, it tends to confirm rather than overturn. Experiment data (from Anthropic research): Same model does "generate → self-review": ~12% error-correction rate A separate model instance reviews: ~37% error-correction rate A different model family reviews: ~52% error-correction rate 1.3 The Value of Independence First principle of quality assurance: the checker must be independent of the generator. In agent architecture, the engineering expression of this is

The Pain : You search GitHub and find everyone using different memory stores — ChromaDB, PostgreSQL, plain Markdown files, "SQLite is good enough for a decade." Which is right? The Answer : All of them. And none of them. Choosing without understanding your scenario is like buying a car without checking the road. 1. The Counter-Intuitive Question: Does Your Agent Actually Need "Memory"? When I was building a university admissions data scraper (91 universities), I made the classic mistake: I equipped the agent with a full ChromaDB + vector retrieval stack, spent three days tuning it, and then discovered — 95% of the agent's time was just reading "which page did I get to last time." A boolean would have sufficed. I built a vector database capable of semantic search. An engineer friend at ByteDance told me their internal agent platform found, after six months, that vector retrieval accounted for only 3.7% of all Memory Store requests. The remaining 96.3%? Key-value lookups, state reads/writes, error dedup. The vector search you spent two weeks integrating might serve less than 4% of your queries. So before discussing "which storage," we must answer a more fundamental question: what does your agent actually need to remember? I categorize memory into four types: Memory Type Typical Content Size Access Frequency Consistency Session state "Processing university 37/91" ~100 bytes Every call Strong Domain knowledge "A-University rate limit is 10 req/s" ~1KB On demand Eventual Error history "B-University returns 403 because UA blocked" ~MB Before new task Append-only Semantic memory "Map 'that red button' to settings page" varies Occasional Eventual See the pattern? If your agent mainly does multi-step automation (data scraping, report generation, CI/CD pipelines), the first three types are the real needs — and none of them require a vector database. Core thesis: memory selection is about finding the layer that is "just enough." One layer too many is waste; one layer too few i

The Pain : You spent an afternoon tuning your agent. Next morning, it stares at you blankly — as if yesterday never happened. What You'll Learn : The 4-stage evolution (Prompt → Context → Harness → Loop), and a runnable 50-line Loop Agent that persists memory. 0. Prerequisites Python ≥ 3.10 pip install openai (openai ≥ 1.0.0) OpenAI API Key OS: macOS / Linux / Windows WSL Goal : Copy-paste the code, run it, and see a Loop Agent that doesn't forget. 1. The Pain: Why Does Your Agent Forget Overnight? At 2 AM, you finally got that multi-step workflow working. The agent followed your carefully designed prompt — data fetching, cleaning, analysis, charting. You close your laptop, satisfied. Next morning, you open the conversation full of hope — and the agent looks at you blankly, as if none of it ever happened. You check the logs. No errors. No exceptions. The agent regenerated everything — it just "forgot" where it stopped yesterday. This isn't a joke. It's the nightmare every serious Agent developer experiences. The root cause isn't "the model isn't smart enough." It's a more fundamental fact: your agent was never designed to survive the night. 2. Four-Stage Evolution: Prompt → Context → Harness → Loop 4-Stage Evolution — each stage solves the previous flaw but adds its own constraint. To understand this, let's use a simple evolution framework: Stage What You Do Fatal Flaw Prompt Engineering Write task description, examples, format into prompt Any unexpected input crashes output Context Engineering Stuff history + intermediate results into context window Token cost grows linearly, hits window limit Harness Engineering Add tool calling, structured output, error capture Framework built, but agent is still "one-shot" Loop Engineering Build closed loop: state + memory + feedback + retry + persistence True engineering — agent starts to "live" Loop Engineering isn't a rejection of Prompt Engineering — it's a transcendence. Prompt still matters. But it's the engine, and you ca

We've all been there: staring at a blood test report from three years ago, trying to remember if that "slightly elevated" glucose level was a one-time thing or a trend. Our health data is scattered across messy PDFs, fitness tracker exports, and physical medical folders. In the era of AI, why are we still manually digging through folders? 📂 Today, we are building the Ultimate Personal Health Knowledge Base . By leveraging Retrieval-Augmented Generation (RAG) , we will transform fragmented medical reports and logs into a searchable, private, and intelligent second brain. We’ll be using LlamaIndex for orchestration, Unstructured.io for parsing those pesky PDFs, and ChromaDB for local vector storage. If you're looking for advanced architectural patterns or production-grade data engineering strategies beyond this tutorial, I highly recommend checking out the deep dives over at WellAlly Tech Blog , which served as a major inspiration for this build. 🚀 The Architecture 🏗️ The goal is to create a pipeline that ingests raw data, vectorizes it, and allows for Hybrid Search —combining semantic meaning with keyword precision (crucial for medical terms!). graph TD A[Raw Health Data: PDFs, CSVs, MD] --> B(Unstructured.io Parser) B --> C{Chunking & Cleaning} C --> D[Sentence-Transformers] D --> E[(ChromaDB Vector Store)] F[User Query: Is my cholesterol improving?] --> G[LlamaIndex Query Engine] E <--> G G --> H[LLM: Local or OpenAI] H --> I[Actionable Health Insight] Prerequisites 🛠️ To follow along, you’ll need a Python environment with the following stack: Unstructured.io : To handle "dirty" PDF and image-based reports. ChromaDB : Our lightweight, open-source vector database. Sentence-Transformers : To generate local embeddings without sending data to the cloud. LlamaIndex : The glue that connects our data to the LLM. pip install llama-index chromadb unstructured sentence-transformers llama-index-vector-stores-chroma Step 1: Ingesting Messy Medical Reports 📄 Medical reports are

How a branch-free loop and byte-space arithmetic let GitHub case-fold every byte of code search at >45 GiB/s on a single core. The post Don’t stop early: Case-folding source code at memory speed appeared first on The GitHub Blog .

I've spent years working with backend systems. APIs, databases, caching, integrations, performance, authentication, cloud infrastructure — these are the kinds of problems that become familiar after you've been building software for a while. Recently, I've been spending more time learning another side of software engineering: LLMs, RAG, AI Agents, tool calling, and MCP. At first, it felt like entering a completely different world. New terminology. New frameworks. New architectural patterns. And honestly, it made me feel like a beginner again. But the deeper I went, the more I noticed something interesting: AI engineering has a lot more backend engineering in it than I initially expected. The Traditional Backend Mental Model A simplified backend architecture might look like: Client ↓ API ↓ Business Logic ↓ Database ↓ Cache / External Services Of course, production systems have much more around this: Authentication Authorization Logging Monitoring Queues Caching Load balancing Rate limiting Distributed systems Cloud infrastructure But the general flow is deterministic. The application receives a request. Our code decides what happens. The application returns a response. Then we add AI. Adding an LLM Looks Easy The first architecture is surprisingly simple: User ↓ API ↓ LLM ↓ Response Send a prompt. Get a response. Done. For a prototype, this can be enough. But production applications rarely stay this simple. Suppose we're building an assistant that answers questions using internal company documents. Now we need retrieval. Enter RAG A simplified Retrieval-Augmented Generation pipeline might look like: Documents ↓ Chunking ↓ Embeddings ↓ Vector Database Then, when the user asks a question: User Question ↓ Embedding ↓ Vector Search ↓ Relevant Documents ↓ Context ↓ LLM ↓ Answer Conceptually, this is easy to understand. But implementing it properly raises a lot of questions. How should we chunk documents? A fixed number of characters? Tokens? Paragraphs? Sections? Semantic

Most "data lineage" you've seen answers a schema question: table B comes from table A , or column B.total comes from columns A.price and A.qty . That's genuinely useful, and tools like OpenLineage do it well. But notice what it doesn't tell you: it says which columns can influence an output. It never says which values actually did . That gap is the whole subject of this post. I built a small, self-contained reference pipeline that captures provenance at the row and field level — "the value in this destination row, this field, was computed from these specific source (row, field) pairs" — and I want to walk through two things: why you'd ever want provenance at that granularity, and why it's genuinely hard once you commit to it. Repo (dbt-core + DuckDB, no server, no cloud, runs on a clean checkout): https://github.com/stevenblough/row-level-prov The one distinction everything follows from Here's the sentence the entire project turns on: Column-level lineage is a schema-sized, static fact you can derive from code. Value-level provenance is a data-sized, dynamic fact you must capture at execution. Put it in complexity terms and the consequences become obvious: Column lineage is O(schema) . It scales with how many columns you have. You can compute it by parsing SQL, offline, without ever looking at a single row. Value provenance is O(rows × fan-in) . It scales with your data volume times how many source values feed each output value. It does not exist anywhere until the query runs, and it can only be captured there , piggybacked on the query that actually produced the values. You cannot "reconstruct" value provenance later by re-querying the sources — the moment the source changes, you'd reconstruct a different answer than what really happened. That single exponent change ( schema → rows × fan-in ) is why value-level provenance has an entire class of problems that column lineage never faces. Why bother? The reasons for this level of granularity Granularity is expensive,

According to a 2024 vFunction survey, over 50% of companies watch more than a quarter of their entire IT budget vanish into technical debt. This isn't happening because developers are bad at writing code. It's happening because our architectural boundaries are eroding. We're letting business rules bleed into database models, and HTTP controllers bleed into core logic. The primary culprit? Modern framework marketing and the "tutorial culture" that dictates how we learn to code. Here's a deep look into why your architecture is bleeding, the engineering principles we're violating, and how Hexagonal Isolation offers a structural cure — with a full working TypeScript codebase, FlowBank , so you can see it applied rather than just described. 1. The Trap: "Time-to-Hello-World" vs. Architectural Debt Most developers learn to build applications through 15-minute video tutorials or framework quickstarts. To win adoption, frameworks optimize for rapid onboarding — showing you how to build a Todo app in three steps using "magic" shortcuts. This creates three architectural traps: The Active Record Trap — ORMs couple database schemas directly to business entities. Change your table, and your core business model breaks. The Controller-to-DB Shortcut — Framework boilerplate encourages writing validation, business rules, and SQL directly inside HTTP handlers. The "Magic" Annotation Bleed — Framework-specific decorators injected straight into domain objects make the entire application hostage to that vendor. Follow these shortcuts long enough and you accumulate Architectural Technical Debt (ATD) . Three years later, upgrading your framework or switching database providers means rewriting the foundational business rules of the company. 2. The Solution: What Is Hexagonal Isolation ? Originally coined by Alistair Cockburn as Ports and Adapters , Hexagonal Architecture solves layer bleed by enforcing strict structural isolation. [External World] ----> ( Port / Interface ) ----> [Core Bus

TL;DR I designed a proposed AWS Transform Custom community skill that prepares Glue 2.0, 3.0, and 4.0 repositories for Glue 5.0. It separates safe mechanical transformations from changes that require human evidence, generates a migration report, and preserves already-compatible files unchanged. Because I didn't have live atx access, the benchmarks in this post are explicitly labeled manually simulated, not agent-executed. The proposal is open as issue #75 — not yet merged, not yet a pull request. The missing data-engineering transformation AWS Transform Custom can apply agent-driven code transformations across a single repository — or thousands of them at once, via AWS Batch and Fargate. As of July 30, 2026, its public sample repository, aws-samples/aws-transform-custom-samples , contained three community-contributed transformations: an EKS version-upgrade-readiness skill, a JBoss-to-Spring-Boot migration, and a Kubernetes readiness migration. None of them touched data engineering. Given that most of my day-to-day work sits across AWS data engineering, Databricks, and Delta Lake, that gap was the obvious thing to fill. What an AWS Transform Custom "skill" looks like Before writing anything, I studied the deepest existing example, jboss-to-springboot , since the pattern it establishes is effectively an unwritten spec for the other two skills as well: README.md — the problem, what the skill does, and how to invoke it via the atx CLI. This is also where the repo draws a clear line: these are readiness transformations. They modify repository artifacts — code and infrastructure-as-code — but they don't deploy jobs, call AWS APIs to change running resources, or claim data-level equivalence. That distinction matters throughout everything below. SKILL.md — the agent-facing definition: YAML frontmatter with trigger keywords, an Objective, explicit Non-Goals , Constraints, worked before/after examples, a "signal in source code → reference file" routing table, and a numbered V

"The real magic of a HashMap isn't that it stores data. It's that it knows where to start looking." In the previous article, we learned that a HashMap organises information around unique keys. Instead of searching every stored object one by one, it uses the key to retrieve information quickly. That naturally raises another question. "If millions of objects are stored inside a HashMap, how does it know where to begin?" Surely it isn't remembering the location of every object individually. The answer lies in one of the most important ideas in computer science: Hashing. Don't worry if the word sounds intimidating. Despite its name, the idea behind hashing is surprisingly simple. Imagine a Huge Apartment Building Suppose you're visiting a friend who lives in a building with 5,000 apartments. If nobody told you the apartment number, what would you do? Probably something like this. Apartment 1 ↓ Apartment 2 ↓ Apartment 3 ↓ ... ↓ Friend's Apartment That would take a long time. Now imagine your friend simply tells you: Apartment 1842 Suddenly, you don't search the building. You walk directly to Apartment 1842. The apartment number isn't your friend. It simply tells you where to begin. Hashing works in exactly the same way. Keys Need Locations Suppose our application stores customers. Customer ID → Customer 1001 → Alice 1002 → Bob 1003 → Charlie 1004 → David The system needs a way to answer one question. "Where should Customer 1002 be stored?" Searching every location first would defeat the purpose of using a HashMap. Instead, the system calculates where that key should go. Notice something important. It doesn't compare Customer 1002 against every other customer. It calculates a location directly. Think of a School Locker System Imagine a school with thousands of students. Every student receives a locker. Student ID ↓ Locker Number ↓ Locker Students don't spend every morning searching hundreds of lockers. Their Student ID determines where they should go. The locker number is

You ask an AI a question. It answers in fluent, confident prose — complete with a study, a percentage, and a name. Some of it is wrong, and nothing about the wording tells you which part. That's the whole problem with hallucinations: the errors wear the same suit as the facts. The fix is not "trust it less" in some vague way. The fix is a repeatable audit step between AI wrote it and I used it . Below is a short hallucination checker prompt you can copy right now, a test run showing what it catches and what slips past it, and an honest account of where a one-liner stops being enough. What counts as an AI hallucination? Not every mistake is a hallucination. A useful working definition: a hallucination is a claim the model states as fact that has no grounding in reality or in your source material. The common shapes: Fabricated citations — a named study, expert, or paper that doesn't exist. Often dressed with a year and an institution. Plausible-but-wrong specifics — dates, version numbers, statistics that are almost right, which makes them worse. Confident category errors — mixing up two similar things (a library and a framework, one company's product and another's). Invented consensus — "experts widely agree that…" with no experts attached. The dangerous ones are the middle two. Obvious nonsense filters itself; a wrong year in a fluent paragraph does not. The copy-paste hallucination checker prompt Here is the short version, free, no strings. It works on ChatGPT, Claude, or any capable model — paste it into a fresh chat, then paste the answer you want audited: Audit the text below for hallucinations. Do not add new information. 1. Extract every factual claim as a separate numbered line. 2. Label each claim: VERIFIABLE (state how to check it), SUSPECT (state what makes it doubtful), or FABRICATION-PATTERN (named source/study/number with no citation). 3. Flag every name, number, date, and citation for manual checking. 4. Finish with the 3 claims most likely to be wrong

Many students graduate knowing how to code. Very few graduate knowing how to engineer software. That's the uncomfortable truth most Computer Science students discover only after facing their first real interview—or worse, after joining their first job. Every year, thousands of students complete coding challenges, solve hundreds of LeetCode problems, build flashy portfolio websites, and proudly call themselves software engineers. Yet many of them struggle when asked questions like: How would you design this system? Why did you choose this database? How would this application scale to one million users? What happens if the server crashes? How would you secure user data? Suddenly, writing code isn't enough. Because software engineering has never been just about writing code. The Biggest Misconception Many universities unknowingly teach students that success in software engineering equals learning programming languages. Students spend years learning: C C++ Java Python JavaScript Then they learn frameworks: React Node.js Express Spring Boot Django Eventually they believe: "I know React and Node.js. Therefore, I'm a software engineer." Unfortunately... That's only one piece of the puzzle. Programming is a tool. Software engineering is a discipline. Those two are related—but they are not the same thing. Coding Is Like Learning to Write Imagine someone learns English. They memorize grammar. They improve vocabulary. They know punctuation. Does that automatically make them a great author? No. Because writing books requires far more than knowing the language. Software engineering works exactly the same way. Programming languages are simply the language engineers use to communicate with computers. Engineering begins after the syntax ends. Software Is Built Long Before Anyone Writes Code Professional engineers don't immediately open VS Code and start typing. Instead they ask questions. Lots of questions. What problem are we solving? Who will use this product? What happens when t

AI takes the path of least resistance. That one characteristic explains most of what changed for me about architecting a system once an agent was in the loop. It is genuinely faster than I am on frameworks, patterns, and the standard way to wire something up . Since it has read more on them than I have. But "least resistance" means it optimizes for the thing in front of it, e.g., getting an endpoint to work or a test to pass. It cannot optimize for the shape the system needs for your use case because it does not know all details. You still own 100% of that part. Two ways least-resistance goes wrong Left alone, the path of least resistance breaks in two opposite directions. It cuts a corner to make the immediate thing work: collapses a boundary, hardcodes a value, skips the seam that would have let two pieces move independently later. And when you try to correct it, it will over-engineer and reach for patterns, layers, and abstractions you did not ask for and don't need yet. Both come from the same place: it is solving the prompt, not steering the architecture . Here is the version I lived with. My app is layered the usual way: an API layer, a service layer under it, a data-access layer under that, with clear rules about what each one is allowed to do. Database transactions belong in the service layer. The agent kept ignoring that. Commits I had scoped to the service layer kept turning up in the data layer, or up in the API. The worst one was a transaction that opened in the service layer and got committed two layers down. If left at simple prompts, it will run and deliver you something that works, but you'll find that along the way it has quietly broken the boundary and created a brittle system. Here is the flip side, from just the other day. I was working on a bug fix with the agent on its own branch off main. Mid-test, I hit a separate gap, related to the feature but not the bug, and asked the agent to fix that too. It sensibly put the gap on its own branch, but b

Building an AI Operating Layer Episode 1 Why I Didn't Start Sooner Most engineering projects begin with an idea. This one began with a question. For months I found myself watching the explosion of AI tools, frameworks, models, and agent platforms. Every week there seemed to be another breakthrough, another library, and another opinion about where everything was headed. I could have started building immediately. Part of me thought I should have. But I realized something, and it kept bothering me. I wasn't afraid of writing code. I was afraid of solving the wrong problem. When a new technology appears, it's easy to jump straight into implementation. Pick a framework. Choose a model. Build something. Ship it. I didn't want to start there because I had a feeling there was a much bigger picture that I wasn't seeing yet. So I waited. I spent my time reading, experimenting, asking questions, and trying to understand how all of these pieces connected. The more I learned, the more I realized I wasn't actually interested in building another AI application. What fascinated me was the system behind the systems. What happens when you stop looking at models, memory, orchestration, tools, policies, and execution as separate ideas and start seeing them as parts of a much larger ecosystem? That question became the beginning of this project. This isn't a story about predicting the future. It's a story about trying to understand it. I'm sure some of my assumptions will be wrong. I'm sure parts of this architecture will change. If they do, you'll see that too. I don't want this journal to only show the polished results. I want it to capture the discoveries, the wrong turns, the redesigns, and the moments where a better idea replaces an old one. At the center of this journey is a project I'm calling the AI Operating Layer. Today it's mostly architecture, documentation, research, prototypes, and a growing collection of ideas. Maybe that's exactly where projects like this should begin. I'

Dependabot keeps your dependencies current, but its defaults can flood your repository with pull requests. Here's how grouping updates, slowing the cadence, and keeping security fixes fast cut the noise on a Microsoft open source project. The post Tame Dependabot: Group your updates, slow the cadence, keep security fast appeared first on The GitHub Blog .

"The best software engineers don't begin by choosing data structures. They begin by understanding what the system needs to do." In the previous article, we learned that data structures never stopped being important after DSA. Their role simply changed. During coding interviews, we often ask ourselves: "Which data structure will solve this problem efficiently?" In Low-Level Design, experienced engineers ask a different question: "What behaviour should this system optimise?" At first glance, these questions sound similar. In reality, they lead to completely different ways of thinking. This article is about understanding why behaviour—not implementation—is where every good design begins. Why Beginners Often Think About Data Structures Too Early Imagine someone asks you to design an online food delivery platform. Many beginners immediately start thinking: Should I use a HashMap? Will I need a Queue? Should I store everything in a Tree? Would a Graph be useful? These aren't bad questions. They're simply being asked too early. Before choosing any data structure, we need to understand what the system is actually expected to do. Software engineering isn't about selecting tools first. It's about understanding problems first. Every Software System Is Really a Collection of Behaviours Let's consider a food delivery application. From a user's perspective, it looks like this. Customer Places Order │ Restaurant Accepts │ Assign Delivery Partner │ Track Delivery │ Order Delivered It looks like one workflow. But an engineer sees something very different. Each step represents a different behaviour. Let's break them apart. Behaviour 1 — Retrieve Existing Information A customer opens an order they placed yesterday. Customer ↓ Order ID ↓ Retrieve Order The system already knows exactly which order it needs. The challenge is retrieving it quickly. Behaviour 2 — Choose the Best Candidate A restaurant has multiple delivery partners nearby. Available Drivers ↓ Choose Best Driver ↓ Assign Ri