标签：#codereview

PR volume went up, ticket quality didn't, and the gap got filled with LLMs on both sides of the review: bots reviewing, bots replying, bots occasionally arguing with bots about priorities that only existed in a teammate's head. Our CEO named the actual problem, and it's bigger than code review.

The call-order change came back pass-with-risk. I read the recommendation, saw it had a name and a reason, and felt the task close. Then I looked at the row under it. How was this verified: not run. Nobody had run the queue. I had a label. I did not have proof. This is Part 6 of The Contract Think produced a brief. Plan produced a gate. Build executed inside it. Review scored every requirement against a verdict instead of an impression. Review reads the diff and the plan and decides whether one satisfies the other. It does not run the queue. It cannot. Its whole job is judgment about what the code should do. Test is where someone finally checks what the code actually does. I had been treating those two as the same step. They are not. Test asks one question, and a verdict is not the answer For every active requirement, Test asks how it was verified. Command run, manual QA, or a comparison against known-good output. One of those three, or a written reason none of them ran. Not a recommendation. Not a risk level. Evidence. I built the matrix against the plan's requirements and filled in each row. Most had a command behind them. The call-order requirement had nothing. The cell read not run, and it sat directly below a pass-with-risk that already carried a name and a reason. That name had almost been enough for me. A named risk feels handled. It is not. It is a risk with a label on it, waiting for someone to actually look. So I ran the queue Three notifications, all with a real reason to fire within the same tick. The scheduler picked them up and ordered them by priority instead of arrival. Two landed in the sequence the requirement wanted. The third jumped ahead of a lower-priority notification that was still mid-processing. The change worked almost every time. Under one timing condition, it did not. That is the gap a verdict cannot see. Review had marked the requirement partial because the wording left the mechanism open. Running it found a real failure inside the mech

CCA-Audit — open source (MIT) AI code review has a trust problem, and it's not that it misses bugs. It's that it invents them. If you've run an LLM over a diff, you've seen it: a "possible null dereference" on a value that's guarded three lines up. A "SQL injection" your ORM already parameterizes. A "race condition" that can't happen. And then — worse — it confidently rewrites working code to "fix" the thing that was never broken. The real bug, meanwhile, sits quietly in the noise. The problem isn't intelligence. It's that most AI reviewers report their first impression as a verdict. A model reads a diff, pattern-matches "this looks like X," and emits a finding — without ever going back to check whether X is actually reachable in this code. Humans do a second pass ("wait, is price validated upstream?"). Most AI-review pipelines skip it. Here are two gates that add that second pass — and a stress test showing what they catch. Gate 1: verify findings before you fix (anti-hallucination) The idea is simple: no finding is allowed into the fix plan until a separate step re-checks it against the real code. After the auditors produce findings, a verification pass takes each one and asks three questions: Does the issue actually exist at the cited line? Is it in the code that changed, or a pre-existing thing outside the diff? Is the stated impact real, or already mitigated elsewhere — a guard upstream, a value validated before this point, a config defined in another module? The key design choice: bias the verifier toward refuting. A wrongly-confirmed finding causes a needless (sometimes harmful) fix; a wrongly-dropped one is cheap to recover. So when the evidence isn't clear, drop it or escalate to a human — don't fix on a hunch. This one step kills the majority of hallucinated findings, because hallucinations rarely survive contact with "show me the exact line, and prove the impact can occur." Gate 2: prove the fix maps to the finding (anti-regression + provenance) Catching

Bolting an LLM onto your pull requests is a weekend project. Building AI code review that your engineers don't disable within two weeks is the actual problem. The failure mode isn't missing bugs — it's crying wolf. Post twenty nitpicks and three hallucinations on someone's PR and they'll mute the bot forever. This is the pipeline we built on Mattrx to earn — and keep — that trust. Mattrx is our multi-tenant marketing-analytics SaaS: ~95k lines of C#, 11 engineers, and enough pull requests that senior-reviewer time was the bottleneck. We tried the naive thing first — pipe the changed file into a model, post the output — and watched the team stop reading it in nine days . TL;DR Dimension Human-only / naive AI (before) AI review pipeline (after) Coverage selective / whole-file dump every PR, diff-focused First-review latency ~6 hours (wait for a human) ~3 minutes (AI first pass) Context none / a naked file diff + call sites + conventions Reviewers one mega-prompt specialized dimensions, in parallel False positives ~35% (so it gets ignored) ~6% (adversarially verified) Merge control human, or nothing severity gate; human always decides Governance none gateway: audit, cost, secret redaction ~90 PRs/week across 11 engineers; the pipeline reviews 100%. First-pass review latency 6h → 3 min. False-positive rate ~35% → ~6% — the single number that decides whether the bot lives or dies. Escaped defects to production down ~40%; senior-reviewer time down ~30%. ~$0.05 per PR (cheap model for style, frontier only for correctness). The one mental shift: AI code review is not about finding issues — models find plenty. It's about not crying wolf . The product is trust, and trust is a false-positive-rate problem. Verify before you comment; let the AI propose and the human dispose. The naive approach — and why it collapses // BEFORE: dump the whole changed file into one prompt, post whatever comes back. foreach ( var file in pr . ChangedFiles ) { var text = await File . ReadAllTextAsyn

Book: The Complete Guide to Go Programming Also by me: Hexagonal Architecture in Go — the companion book in the Thinking in Go series My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub You open a pull request in a Go repo you just joined. Ten minutes later there are six comments on it. None of them are about your logic. They point at a capitalized error string, a receiver named this , a context stored on a struct. Each comment links the same page: the Go Code Review Comments wiki. That page is the closest thing Go has to an official style council. It grew out of the comments Go's own maintainers left on CLs for years, and most Go teams treat it as the default rulebook. The problem is that the wiki tells you what but rarely why , so the rules read like arbitrary taste. They aren't. Each one exists because the alternative bit somebody. Here are ten that reviewers cite the most, with the reason behind each. Everything below is idiomatic on Go 1.23+. 1. Error strings are lowercase and unpunctuated The rule: an error string should not be capitalized and should not end with punctuation. // wrong return fmt . Errorf ( "Failed to open config." ) // right return fmt . Errorf ( "failed to open config" ) The reason is wrapping. Go errors get concatenated. Your string is almost never the whole sentence a user reads; it's a fragment in a chain built with %w : return fmt . Errorf ( "load settings: %w" , err ) // -> "load settings: open config: permission denied" Capitalize your fragment and you get load settings: Open config: permission denied in the middle of a line. End it with a period and you get a period in the middle of a longer message. Lowercase, no trailing punctuation, and every fragment composes cleanly no matter where it lands in the chain. The exception is a string that begins with an exported name or acronym, which keeps its own case ( HTTP , TLS ). 2. Receiver types are consistent ac

You merge the PR. The build passes. The code does what you expected it to do. You move on. That is review for most engineers. A final read. A feeling that things looked right before the branch closed. I did it the same way for years. Three phases had already run before this one. Think had scoped the work, Plan had written the requirements, Build had shipped a diff that matched the plan exactly. I trusted that the chain held. I had never actually checked. Then I ran the Review phase, and checking turned out to mean something specific: not does this work, but does this requirement hold up, and what is my evidence. I went in expecting to approve it or send it back. The phase gave me three answers instead: covered, partial, missing. I found out what they meant one requirement at a time, starting with the one I almost got wrong. I had been giving impressions, not verdicts The notification scheduler used a queue to manage dispatch. Every call to the external provider went through it. The provider was never exposed directly. The requirement said the provider must be notified. It was notified, exactly the way I had pictured it. I almost called it covered and moved to the next line. The Review phase stopped me there. But the requirement said must be notified , not how. The queue had introduced a call order and a timing the requirement never anticipated. Nothing was broken. Something had changed shape, quietly, and nobody had written that shape down. I sat with that for longer than I expected to. Not because the code was wrong. Because I could not immediately tell you whether the change mattered. The same pass gave the shim from Plan a different verdict on the same page: covered. Mapped to the requirement it existed to satisfy, no gap between what was promised and what was in the diff. One requirement held exactly the shape it was given. The other had quietly grown a new one. Same review. Same pass. Two verdicts. Partial is not a softer word for broken. It is the verdict for

Originally published on productize.life . Quick answer: gstack is an open-source (MIT) skill set that Garry Tan, president of Y Combinator, builds with every day. It turns Claude Code into a team of 23 specialists, CEO, engineers, designers, QA, and a release engineer, forcing every change through a multi-lens review before shipping. The point is not speed; it is taste written into software. Last week I was going through a repo that collects skills for coding, several of them. Most share one theme: helping AI write code in a systematic way, and faster. But one made me stop longer than the rest, called gstack, for two reasons. One: its owner, Garry Tan, president and CEO of Y Combinator, took the stack he actually builds with every day and opened it for free. Two: it does not sell "code faster," it sells "review before you ship." Once I actually opened it, it was not just a toolbox but one of the clearest examples of an idea I have been interested in for a while. On the day AI can write code very fast, the bottleneck of the work is no longer speed. I will tell it in three parts, starting with what it is , then what gstack believes , and closing with lessons for people who build products, not just people who write code . Terms, gathered here in one place agentic coding letting an AI agent run the coding work in its own steps, from planning to writing to review to shipping, not just autocompleting a line at a time. skill a packaged set of instructions an AI agent (like Claude Code) can call, like a shortcut that wraps one way of doing one thing. review lens reviewing one piece of work from several roles, for example as a CEO, an engineer, a designer. taste the sense and judgment of what is good and what is bad, what to build and what not to ship. The part that is still human. Part 1: What gstack is Garry Tan describes gstack in the README plainly, as the way he works. "It turns Claude Code into a virtual engineering team: a CEO who rethinks the product, an eng manager

You ask Claude to review your code. It says "looks good, clean, well factored". Of course it does. It wrote that code five minutes ago. You just asked the author to grade his own paper, and he gave himself an A. Having an AI review code works. But not by asking the one who just wrote it. Quality doesn't come from a smarter model, it comes from an architecture where no role checks itself. The self-preference bias This isn't a hunch, it's measured. A model evaluating its own output rates it higher than others' at equal quality: the self-preference bias , documented by Panickssery and co-authors in 2024, and it's causal, not correlational. The model recognizes its own style and prefers it. In practice that means the naive loop "write, then review what you just wrote" is broken by construction. You don't get a review, you get a justification. The agent already decided its code was good the moment it produced it; asking again only confirms. The blind reviewer So the first rule: the reviewer is never the author. In my config, the review agents run in a clean context . They don't see the implementation prompt, they don't know what constraints the author set, they meet the diff like a colleague on Monday morning. And when the author is a known model, the reviewer is from a different family , to break style recognition. One detail matters as much as the rest: the developer's name never enters the reviewer's prompt. No "this was written by a senior", no "review this model's work". The author's identity is exactly the information that triggers the bias. We take it off the table. No finding without a receipt The second trap is the opposite of the first. An AI reviewer, especially in a clean context, tends to over-flag: it invents problems to look useful, it flags "vulnerabilities" that aren't. A review that cries wolf on every line is no better than a complacent one: either way, you stop listening. Hence the receipt rule. Every finding must cite a file:line and pass a check bef

Revisar código es una de las actividades más subestimadas del desarrollo de software. La mayoría de los equipos la tratan como un trámite, algo que hay que aprobar antes de mergear. El resultado es que los PRs se aprueban con un “LGTM” después de dos minutos de scroll, y los problemas reales pasan de largo. Una revisión bien hecha no es leer línea por línea buscando typos. Es entender qué intenta hacer ese código, si lo hace de la manera correcta, y si introduce riesgos que no existían antes. Eso requiere un proceso, no un instinto. Este artículo cubre cómo estructurar ese proceso: qué revisar, en qué orden, qué preguntas hacer, y cómo detectar problemas de seguridad sin ser un experto en ciberseguridad. Empieza por el contexto, no por el código El error más común en code review es abrir el diff y empezar a leer desde la primera línea modificada. Antes de ver una sola línea, necesitas entender qué problema resuelve este cambio. Lee la descripción del PR. Si no hay descripción, o si dice “fixes bug”, ya encontraste el primer problema. Un PR sin contexto obliga al reviewer a reconstruir el razonamiento del autor desde cero, y eso aumenta la probabilidad de aprobar algo que no debería aprobarse. Lo que un buen PR debe explicar: Qué cambia no cómo, sino qué problema resuelve Por qué este approach si hay alternativas que se descartaron, decirlo Cómo probarlo, pasos para verificar que funciona Qué no cubre, scope explícito evita confusiones Si tienes esa información antes de ver el diff, tu revisión va a ser significativamente más efectiva. Qué revisar y en qué orden No toda línea de código merece el mismo nivel de atención. Un buen reviewer distribuye su energía de forma inteligente. Primero: arquitectura y flujo de datos. ¿El cambio tiene sentido a nivel de diseño? ¿Agrega una dependencia innecesaria? ¿Rompe alguna abstracción existente? Esto es lo más difícil de cambiar después. Segundo: lógica de negocio. ¿El código hace lo que dice que hace? ¿Los edge cases están cub

I think a lot of people still imagine AI coding as opening ChatGPT, asking for code, and copy-pasting the result. That's not really how I work anymore. The biggest shift for me is that planning matters far more than coding. Earlier, execution was expensive, so most of the effort went into writing code. Now execution is cheap. I can have an agent implement something in minutes. The hard part is making sure the plan is correct. Most of my effort goes into thinking through the architecture, edge cases, failure modes, test strategy, and how the change fits into the broader system. If the plan is vague, the agent will confidently implement the wrong thing. The quality of the result is mostly determined by the quality of the plan. Once I have a plan, I break it into small independent pieces. Each piece should be executable without additional clarification. If an agent needs to stop and ask questions, the task probably isn't broken down enough. Those pieces become tickets. Then an agent picks up a ticket and implements it. The important thing is that the agent isn't operating in a vacuum. I try to give it a good environment to work in: Clear architectural rules Reusable skills and workflows Guardrails Hooks for things that must always happen One lesson that really stuck with me is that instructions are guidance, not guarantees. At one point I had "always use a git worktree" written in AGENTS.md. The model still ignored it occasionally. When I dug into it, the answer was simple: models can drift from instructions. So if something absolutely must happen, don't rely on instructions. Enforce it. Put it in a hook, script, validation step, CI check, or some other deterministic mechanism. If it is important, make it impossible to skip. Once the implementation is done, the agent opens a PR. This is where another useful pattern comes in: don't let the same model review the code it wrote. I usually have one model implement and another model review. Different models catch different t

GitHub moved Copilot Chat's richer pull request experience to general availability this week — side-by-side chat with diffs, inline editing, and context-aware answers without leaving the review view. Previously in public preview, it is now live for all Copilot license holders. It is a real improvement for reviewing changes inside a single pull request. But it highlights a gap that per-PR AI tooling structurally cannot close: knowing what is open across the rest of your organisation. The Problem That Lives Outside the PR Most engineering teams don't work in one repository. They ship across services, libraries, and infrastructure — often with related PRs open in multiple repos simultaneously. A reviewer approving a payments service change without knowing that a dependent auth-service PR is still in draft is reviewing without full context. This is not a quality-of-feedback problem. It is a visibility problem. No amount of intelligence surfaced inside a PR tells you what is happening across your repositories. Gartner's 2026 assessment of AI coding agents makes the point clearly: the bottleneck has shifted from generating code to reviewing, securing, and governing it. Better per-PR AI raises the floor on feedback quality. The teams that pull ahead will be the ones who also solve the coordination layer — which PRs are open, which are stale, which are blocked on a dependency in another repo. What Changes With Better In-PR AI GitHub's GA release makes the review experience faster and less disruptive for individual PRs. That matters. But as per-PR intelligence becomes table stakes, the differentiator shifts toward cross-repo awareness: who is waiting for review, what related work is in flight, and where the actual bottlenecks in the delivery pipeline are. Engineering leaders should be watching PR age distribution and review load across all repositories — not just the ones that happen to be open in a browser tab right now. For teams already dealing with multi-repo sprawl, Cod

TL;DR — AI assistants are producing C# that looks correct and passes review, but reintroduces production regressions we spent years training out of teams. I'm trying to find out whether other .NET teams see the same patterns — and what's actually catching them before merge. More AI-generated C# is landing in pull requests. Most of it is fine. But a specific category keeps slipping through — and it's the dangerous one, because it compiles, tests pass, and a human skim says "looks good." The pattern The code compiles. Tests pass. Review approves. Production finds out. These aren't syntax errors. They're architectural intent violations — the kind of thing a senior dev would have caught in review before PR volume tripled. Five regressions I keep seeing 1. EF Core read paths without AsNoTracking() Fine in dev. Expensive on a hot read path in prod. // ❌ Looks reasonable. Tracks entities you never mutate. var orders = await _db . Orders . Where ( o => o . CustomerId == id ) . ToListAsync ( cancellationToken ); Fix direction: AsNoTracking() on read-only queries, or a team convention documented in CLAUDE.md / Copilot instructions. 2. Captive dependency (scoped service in a singleton) Compiles. Runs. Wrong state across requests. // ❌ Singleton lives forever; scoped dependency does not. services . AddScoped < IOrderRepository , OrderRepository >(); services . AddSingleton < ReportCache >(); // ctor takes IOrderRepository Fix direction: align lifetimes, or inject IServiceScopeFactory instead of capturing scoped services. 3. Dropped CancellationToken The method accepts cancellation. The downstream call ignores it. // ❌ Signature honours cancellation; body doesn't. public async Task RunAsync ( CancellationToken cancellationToken ) { await Task . Delay ( 500 ); // overload with token exists } Fix direction: forward cancellationToken to every downstream async call that accepts one. 4. Swallowed exception Failure disappears. Monitoring stays green. // ❌ "Handle errors gracefully" —

This article was originally published on aicoderscope.com Most AI coding tools are generalists—they write code, answer questions, and somewhere in the feature list, review pull requests. CodeRabbit is the opposite: one thing, done obsessively. Every feature, every design decision, every pricing tier revolves around making PR review better. After reviewing the pricing, benchmarks, and comparing it to GitHub Copilot's native code review, here's the honest assessment. What CodeRabbit actually is (and what it isn't) CodeRabbit sits between your developer's git push and the merge button. You connect it to your repository host—GitHub, GitLab, Azure DevOps, or Bitbucket—and it automatically reviews every pull request. No button to click. It reads the diff, checks it against your full codebase for context, runs 40+ static analysis tools, then uses a multi-model AI stack to flag bugs, security issues, and style violations directly in PR comments. What it cannot do: generate application code, scaffold features, or replace a coding assistant. It is review-only. That constraint shapes everything about the product. At $40M ARR as of April 2026 (up 700% year-over-year from $5M ARR in April 2025), with 2 million repositories connected and more than 13 million pull requests reviewed, CodeRabbit has clearly found a market. It currently holds the #1 position among AI apps on GitHub Marketplace. How the review actually works Every CodeRabbit review runs in three stages. Stage 1: Context engine. Before analyzing the diff, CodeRabbit indexes your codebase using a retrieval system similar to what backs its code reviews across millions of repositories. It uses NVIDIA Nemotron for this context-gathering and summarization stage—a lightweight open model optimized for retrieval rather than generation. This is why CodeRabbit catches cross-file issues that pure diff-reviewers miss. Stage 2: Static analysis. A deterministic SAST layer runs linters that don't need AI inference: Biome, ESLint, Ruf