标签：#bug

Point a model at your repositories, ask why something broke, and you get an answer. Coherent, names a mechanism, shows you how it got there. The working is the problem rather than the reassurance. It looks the same whether it ends at a value in a record or at what usually happens, and only one of those is evidence. I ran the same question at four levels of evidence, on two systems, across three models. The question was never find me a bug. It was always this already happened, work out how. The pass that worries me is the one just short of enough evidence. There is enough there to build a real hypothesis, and not enough to tell a real one from a plausible one. Two of its guesses sent me to look at things I had not checked. One was a dead end. The other was a real problem I had not known about, and nothing in either answer told me which was which. A good guess costs the same to chase as a real one. You find out which it was at the end. The four rungs, and what each one removes I stopped thinking about this as adding context. Each rung takes away something the model would otherwise have to guess at, and that is the more useful way to look at it. One. Repository access, broad. Point it at everything and ask. It removes nothing. The answer was structurally sensible, named components that were not involved, and arrived with no more hesitation than the correct one did three rungs later. If your team tried this once, got something confident and wrong, and decided the tooling is not there yet, this is probably where you stopped. Two. Three bounded repositories, plus a written map. Which service talks to which, over what protocol, with what delivery and ordering guarantees. This removes rediscovery. What made it work was not the map. It was telling the model to treat the map as true and not go and check. That bought focus and gave up verification. Maps go stale quietly, and I have removed the step where it might have noticed. Three. Plus traces and logs covering real executio

Hard debugging is rarely defeated by a lack of tools. It is defeated by three quiet habits: assuming the fault is where the symptom appears, clinging to the first explanation, and hoping a tool will do the thinking. A difficult fault is usually lost to those habits before you read a line of code. The engineers who resolve hard faults are the ones who notice these defaults and replace them with a patient, evidence-first mindset. Most hard bugs are lost before we touch them, in the attitude we bring to the session. When something breaks, the average person rushes in with three quiet habits: they assume the fault lives exactly where it shows up, they cling to the first explanation their mind offers, and they hope a tool or a smarter person will tell them what to do next. Those habits feel natural, but on hard faults they are exactly what keep us stuck. Put two engineers on the same failing board. One finds a way through in an afternoon; the other is still going three days later. The difference is rarely raw intelligence or how many commands they know. It is the mental posture each brings to the work before the first step. Handling a hard debug session is less about knowing every tool and more about managing your own assumptions, reactions, and impatience. A tough problem is usually lost in your mindset before it is lost in your methods. Habit one: starting too narrow The first habit is to fix on the most visible symptom and refuse to look anywhere else. Something breaks, so we stare at the last thing we changed, and we return to it because it is familiar and close at hand. When the answer is not there, we look harder in the same place instead of stepping back. Here is what that looks like on real hardware. A device keeps dropping off the bus. You are a kernel person, so you open the driver and read it, carefully, for three days: the probe path, the error handling, the power-management callbacks. Every line is correct, and the device still fails. The fault was a layer b

The owner had already asked for the alert emails to stop. A fix shipped. Then another email landed. Then another. "ong it just ssent me abother email," he said, voice-dictated, unedited. Fifteen minutes later: "go another one." The system was reporting an outage that did not exist. The Transport That Only Ever Failed A 14-PR merge train had just moved every cron producer's alerting off shared email and onto Buzz, a Nostr-relay team chat. One producer per PR, each with its own liveness contract and a bead receipt. It shipped cleanly. But the library backing those producers carried a default that had only one job: fail. AF_BUZZ_CMD = " ${ AF_BUZZ_CMD :- af_default_buzz_post } " af_default_buzz_post returned 1 with "no Buzz transport injected". Every caller that sourced the library (which is every cron producer) exhausted its Buzz retries and fell through to the email floor. The system reported a false Buzz outage while the relay was healthy. It did this 2 to 5 times per hour. Evidence arrived in the logs: 581 dedup markers, a steady stream of "[INTENT ALERT FLOOR: Buzz unreachable]" emails, and sweep.log showing buzz=ok only for the handful of callers invoked through the CLI entrypoint rather than by sourcing the library. That asymmetry was the bug. The CLI had a one-line fixup swapping in the real transport, annotated in a comment as "the library path is unchanged". The library path did not, and the cron producers all take the library path. The fix promoted the real transport to the default for both seams. af_buzz_transport already discovers the installed buzz-notify.sh and already fails closed when it is genuinely missing. The dead CLI fixup was deleted. Fail-closed behavior survives, but now it is conditional on genuine absence rather than on every caller remembering to opt in. Why not migrate callers one at a time? Because the per-caller route leaves the next new producer to rediscover this the same way. Flipping the default fixes the class, not the instance. The

Start with the Built-in Inspector Before reaching for external tools, remember Node.js has a built-in debugger. Run your script with --inspect and open chrome://inspect in Chrome to get a full DevTools experience: breakpoints, step-through, console, and even memory profiling. node --inspect app.js For a quick breakpoint without touching the browser, use --inspect-brk to pause on the first line. This is great for debugging startup issues. Use debugger Statements and Conditional Breakpoints Sometimes you need a breakpoint only when a condition is true. Instead of littering your code with if blocks, set a conditional breakpoint in DevTools. Right-click the line number, choose "Add conditional breakpoint," and enter an expression like user.id === 42 . For quick inline debugging, debugger; works but remember to remove it before committing. I often use it temporarily when I'm too lazy to open the DevTools UI. Log Like a Pro with util.inspect console.log of an object prints [object Object] which is useless. Use util.inspect with depth and colors to see nested structures clearly. const util = require ( ' util ' ); console . log ( util . inspect ( myObject , { showHidden : false , depth : null , colors : true })); Or in modern Node, you can use console.dir with { depth: null } for the same effect. Async Stack Traces: Don't Lose the Context Async errors are painful because stack traces often end at the event loop. Node 12+ gives you better async stack traces by default, but you can improve them further by using Error.captureStackTrace in your own error classes. class MyError extends Error { constructor ( message ) { super ( message ); Error . captureStackTrace ( this , MyError ); } } This makes the stack trace point to the caller, not the constructor. Handle Unhandled Rejections and Exceptions Silent failures are the worst. Set up global handlers to log errors properly and exit gracefully. process . on ( ' unhandledRejection ' , ( reason , promise ) => { console . error ( ' U

Originally published on hexisteme notes . Every test passed. The page was in pieces. I was rebuilding a small internal dashboard — FastAPI, Jinja2 templates, hand-written CSS, no build step — and the layout had come apart. Timeline rows unstacked into a vertical column. Status dots floated free of their rows. Log group labels overlapped. It looked exactly like a page whose stylesheet had failed to load. The stylesheet had loaded. All 481 tests in the suite were green. And when I grepped the CSS file for the rules that were obviously not being applied, they were sitting right there on disk, correctly written. The cause was a comment closer. Somewhere in the middle of the file, a /* had been closed with #} — Jinja's comment terminator — instead of */ . Muscle memory, from switching back and forth between .html templates and .css . CSS then did precisely what the specification tells it to do: it kept reading. The comment ran on and swallowed the next 15 lines of rules — the timeline-row grid, the feed, the bucket layout, the dot alignment — until it hit the next real */ , seventeen lines down. No error. No console warning. No failing test. The rules were present in the file and absent from the page at the same time. The typo is the least interesting part. What's worth keeping is why CSS is designed to fail without symptoms, why source-level review cannot see it, and why the fix is a two-line assertion rather than more care. Why nothing complained: CSS has no fatal errors The CSS Syntax specification defines comment consumption like this: on seeing /* , consume everything "up to and including the first */ , or up to an EOF code point." No notion of a comment being too long, no heuristic about blank lines or braces, no upper bound. First */ wins. A comment closed seventeen lines later than intended is not a malformed comment — it is a well-formed comment that happens to be seventeen lines long. The parser has no way to know you meant something else. Even if the comment h

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry. This happened a...

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry. Did anyone ask for...

Deliberately best-effort — a settings tweak failing shouldn’t fail a deployment that already succeeded. It surfaces as public: false in the response and a visible warning in the UI, rather than silently handing someone a broken link. Live Stripe checkout failed on day one. Test mode worked perfectly. Live mode returned: Invalid line_items[0]: the product tax code is missing… Product tax code is required for Managed Payments, which is enabled by default on your account. A whole class of bug that only exists in production. I reproduced it directly against Stripe’s API before touching code, then opted the session out of Managed Payments — rather than inventing a tax classification, since whether to collect sales tax is a business decision, not a code one. The meta-lesson Every one of these five bugs was invisible to the test suite. The unit tests were green the entire time — because they tested my parsing logic, and every bug was in the query I sent or the transport I sent it over. Four of them were only findable by hitting the live endpoint and reading actual output. The 1996–2006 bug in particular looked like a total success from every angle except one: 566 results, HTTP 200, tests passing, correct shape. You had to actually look at the dates. Best Use of Sentry Not submitting to this category — DAREALTYTE doesn’t currently use Sentry. Worth being straight about it, since this project is a decent argument for adding it. Bugs 4 and 5 both returned HTTP 200 with well-formed payloads. Error monitoring wouldn’t have flagged either one; nothing threw. What would have caught them is exactly what I ended up doing by hand — inspecting real production responses and noticing the values were wrong even though the shape was right. The one place Sentry would have paid off immediately is Bug 3. The Safari Load failed was reported to me as a screenshot from a phone, with no stack trace and no way to reproduce it in my own environment. A Session Replay or a captured client-side exce

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . Every developer has experienced that moment when a project works perfectly but doesn't feel perfect. That was exactly what happened while I was building my Gem Price Estimator , a web application designed to estimate gemstone values based on multiple characteristics and pricing rules. The calculations were accurate. The interface looked good. But something bothered me. It wasn't as responsive as I wanted it to be. That small delay was enough to make the application feel slower than it should, and I knew there had to be a better way. This wasn't about fixing a crash or a broken feature. It was about finding the hidden performance bottleneck. The Project The Gem Price Estimator analyses several gemstone properties and combines them to generate an estimated market value. The estimation process considers multiple factors, including: Carat weight Color Clarity Cut Other pricing adjustments Every user interaction triggered a complete recalculation of the estimated value. Initially, this approach worked well while the project was small. As the pricing logic became more sophisticated, however, the application started doing significantly more work than necessary. The First Sign Something Was Wrong Nothing was technically broken. There were no JavaScript errors. No failed requests. No database issues. The application simply felt slower every time users adjusted the estimator. Those tiny delays might seem insignificant individually, but together they reduced the smoothness of the overall experience. I wanted every adjustment to feel nearly instant. That became my goal. Investigating the Problem My first assumption was that the issue was caused by database operations. So I started checking: Database queries Network activity Browser Developer Tools Console logs Individual calculation steps Surprisingly... None of those were the real problem. The application wasn't waiting on the database. It wasn'

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry. ...

Originally published at wostal.eu . TL;DR : I handed a CI re-platform — GitHub Actions → Argo Workflows, GitHub → Codeberg — to an AI agent. The rewrite looked correct and passed lint, but it didn't account for one thing: the runtime environment had changed completely. The old runner was an external VM; the new one is a pod inside the cluster, where Tailscale IPs aren't routable. Nobody ran the pipeline end-to-end, so every difference stayed as a landmine — six of them. The worst part wasn't the AI's mistake. It was that I spent hours blaming the network when the real culprit was a single missing newline in an SSH key. This is the companion to a different incident. While I was migrating my homelab k3s control plane from SQLite to etcd , the pipeline that ran that migration turned out to be broken in ways that taught me more than the migration itself. In this post I'll cover: Where the mess came from — an AI-assisted CI re-platform that nobody validated end-to-end The phantom CNI bug — hours of chasing MTU, Tailscale, firewall, and conntrack The real culprit — a single missing newline in an SSH key Why early failures hide later ones — six latent bugs, surfaced one at a time The lesson about delegating migrations — to a human or an AI Where the mess came from The entire CI had just been migrated from GitHub Actions to Argo Workflows , and from GitHub to Codeberg — and that migration was handed to an AI agent 🙂. The re-platforming rewrote the logic roughly 1:1, but it didn't catch that the runtime environment had changed fundamentally . The old runner (GitHub Actions / ARC) behaved one way; the new one is a pod in the cluster — from which Tailscale IPs aren't routable, but private IPs are. Nobody (human or AI) ran the pipeline end-to-end, so all of those differences stayed as landmines. That's why there were six latent bugs and the hang I'm about to describe — not random faults, but traces of a GHA→Argo translation that was never validated in the new environment. The p

Two people asked a context compressor two completely different questions. It gave them the same answer. Not a similar answer — byte for byte the same 544 characters. Here's what that looked like: query="Fix the IntegrityError on commit" level=L0 -> 159 tok cache_hit=False query="Explain the tax rounding TODO in compute_tax" level=L3 -> 159 tok cache_hit=True identical output: yes (544 chars both) Different question. Different compression level. Same 544 characters, served from cache. Finding it I wasn't looking for this. I was auditing something else entirely — measuring how much meaning a context compressor loses, not how fast it runs. My harness feeds the same corpus through the compressor with different queries and checks which critical substrings survive: file paths, error types, line numbers, identifiers. I noticed two rows in my results table were identical. Same token count, same output. My first assumption was that my own harness had a bug — that I was passing the same query twice and hadn't noticed. So I changed the second query to something with no words in common with the first, and bumped the compression level from L0 to L3, which should change the output dramatically on its own. Same 544 characters. That was the moment it stopped being my bug. The cause One line: sid = content_hash(content) That sid was doing two jobs. It was the shadow ID — the handle used to refer to a stored document. And it was also the cache key. As a shadow ID it's correct: the same content should get the same handle. As a cache key it's wrong, because the output of compress() doesn't depend only on the content. It depends on the content and the query and the compression level. Two of those three inputs were simply not part of the key. So the first caller warmed the cache for a piece of content, and everyone who touched that same content afterwards got the first caller's answer — regardless of what they actually asked for. Why this is worse than a stale cache A stale cache gives y

I have a small script, reply_comments.py , that keeps me from having to re-scan every DEV.to article for new comments by hand. It has two commands: pending (unanswered comments I haven't drafted a reply to yet) and audit (drafted replies I said I'd paste manually but apparently never did). I've already fixed two bugs in this file — one in needs_reply() (a thread stayed "handled" forever after a single reply, even when the other person followed up again) and one in audit() (it only checked direct children, so a reply nested two levels deep was invisible). Today I found a third, in pending() itself, and it's the kind of bug that hides precisely because the first two fixes made everything else in the file look trustworthy. What pending() actually does Comments on DEV.to come back from the API as trees — each top-level comment has a children list, and replies can nest arbitrarily deep. pending() walks each article's top-level comments and decides, for each one, whether it needs a reply: def pending (): try : drafted_text = open ( DRAFTS , encoding = " utf-8 " ). read () except FileNotFoundError : drafted_text = "" drafted_codes = set ( re . findall ( r " ^## (\S+) " , drafted_text , re . M )) out = [] for a in api ( f " /articles?username= { ME } &per_page=100 " ): if not a [ " comments_count " ]: continue for c in api ( f " /comments?a_id= { a [ ' id ' ] } " ): if not needs_reply ( c ): continue if c [ " id_code " ] in drafted_codes : continue out . append ({ " id_code " : c [ " id_code " ], " author " : c [ " user " ][ " username " ], " article " : a [ " title " ], " comment_url " : f " https://dev.to/ { ME } /comment/ { c [ ' id_code ' ] } " , " body " : strip_html ( c [ " body_html " ]), }) return out needs_reply(c) is the fix from a few weeks ago — it recurses the whole subtree and checks who posted the most recent message, not just whether I've ever replied. That part's correct. The bug is in the two lines right after it: c["id_code"] and c["body_html"] . c here i

Learning CSS can feel like magic, but it can also be incredibly frustrating. One minute your website looks perfect, and the next minute, a single line of code breaks the entire layout.If you are struggling to get your web pages to look exactly how you want, don't worry. Here are 5 of the most common CSS mistakes beginners make and exactly how you can fix them. 1. Forgetting the CSS Box Model (Adding Padding Breaks Width) The Mistake : You set a box's width to 100%, but as soon as you add padding: 20px; or a border, horizontal scrollbars appear and your layout breaks.Why it happens: By default, CSS adds padding and borders on top of the width you specified. So, 100% width + 20px padding left + 20px padding right = wider than the screen!The Fix: Always use box-sizing: border-box; at the top of your CSS file. This forces the browser to include padding and borders inside the specified width. /* Add this to the very top of your CSS file */ { box-sizing: border-box; margin: 0; padding: 0; } 2. Confusing Block vs. Inline Elements The Mistake: You try to add a vertical margin, width, or height to a or an tag, but nothing changes on the screen.Why it happens: Tags like , , and are inline elements. By default, inline elements ignore top/bottom margins, heights, and widths.The Fix: Change the element's display property to inline-block or block. /* Fix: This will now respect your width and margin settings */ a { display: inline-block; width: 150px; margin-top: 20px; } 3. Overusing Absolute Positioning (position: absolute) The Mistake: Using position: absolute; to push elements around the screen until they look "perfect" on your laptop, only to find the layout completely scrambled on a mobile screen.Why it happens: Absolute positioning takes elements out of the normal document flow. It makes your website completely rigid and unresponsive.The Fix: Stop using absolute positioning for general layouts. Instead, learn and use CSS Flexbox or CSS Grid to build flexible layouts. /* Inst

On Windows I kept launching background servers from a task runner and watching them die the instant the launching step finished — no error, no log, just gone. The task runner was wrapping everything in a job object, and job-object teardown kills every child process on return. Nothing I did inside the child mattered; its death warrant was signed by how it was born. The workaround was to have the process created by something that outlives the runner — the OS scheduler, a WMI process-create call — instead of spawning it as a doomed descendant. When a process keeps dying without a trace, look at its lineage before its code — some parents kill their children on the way out, and no amount of hardening inside the child fixes how it was spawned.

When I published v2.0.0 , I thought the hard part was over. The CLI could turn Claude artifacts into native Windows applications in a single command. It worked on my machine, the demos looked great, and I was excited to share it. Then people started using it. That's when I realized the difference between building software and shipping software . The bugs only users can find Most of the problems weren't visible in a five-minute demo. Some applications worked perfectly. Others silently lost all of their data after being closed. Some builds succeeded exactly once. The second build failed unless the runtime cache was manually deleted. Everything looked correct... until someone actually tried using it. The rabbit hole What I thought would be a quick patch became a deep dive into things I'd never touched before. Over the course of v3 I ended up learning far more than I expected: Git branching and release workflows Semantic versioning GitHub Actions npm packaging npm pack Runtime integrity verification SHA-256 checksums Neutralino runtime management JavaScript debugging Integration testing None of those were part of the original project idea. Persistent storage finally works This was the biggest challenge. Artifacts using localStorage behaved differently depending on where they were running. Inside Claude everything worked. Inside the generated desktop application... not always. Tracking this down took far longer than writing the original feature. The result is that applications now persist their data correctly across launches without requiring any code changes. If your artifact uses localStorage , it should simply work. Runtime improvements The runtime layer also received a major overhaul. Version 3 now handles: automatic runtime downloads integrity verification cache recovery consecutive builds without manual cleanup Those aren't exciting features to demo, but they're exactly the kinds of improvements that make a tool feel dependable. Better testing I also spent a lot mo

This is part of my Building Fluentic Style series, where I’m writing down the design decisions, tradeoffs, and small surprises from building Fluentic Style . It is one thing to make a styling library feel good in a client-side app. It is another thing to make it feel good in Next.js App Router. In a simple SPA-style development setup, most of the styling loop lives in one place: component renders in the browser Fluentic style chain resolves atomic CSS rule is inserted DevTools can inspect the generated rule sourcemap points back to authored code That is already a lot of work. But at least the browser is the main place where the style is produced and consumed. Next.js App Router changes the shape of the problem. Now the page can involve: server rendering React Server Components client components streamed HTML hydration client-side navigation HMR Webpack or Turbopack development sourcemaps production extraction So the hard part is not just “can Fluentic run in Next.js?” The hard part is: Can Fluentic keep the same CSS debugging experience when styles cross the server/client boundary? That is what this post is about. Docs for the Next.js integration are here: Next.js Integration DevTools And Sourcemaps Runtime And Dev Debug Without Getting Lost The Goal Was Not A Special Next.js API I did not want Fluentic to have one mental model for client apps and another one for Next.js. This should still be normal Fluentic: const card = style ({ padding : 16 , borderRadius : 12 , }). hover ({ boxShadow : ' 0 12px 30px rgb(15 23 42 / 0.16) ' , }); export function Card () { return < section css = { card } > Hello </ section >; } And this should still be normal Fluentic too: const buttonStyles = { root : style . slot ({ display : ' inline-flex ' , border : 0 , }), label : style . slot ({ fontWeight : 700 , }), }; const danger = style . scope ([ buttonStyles . root ({ backgroundColor : ' #dc2626 ' , }), buttonStyles . label ({ color : ' #ffffff ' , }), ]); The Next.js integration shou

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . The build was done. Themis Lex worked on my machine, and not in the "works if you squint" way. A court clerk enters their role, describes their workflow, picks a data sensitivity level, and gets back a PDF with two sections: where AI can safely support the work, and where it must never touch it. Claude via Bedrock generates the assessment. Server-side PDF render. No accounts, no storage, session ends when the download does. Three weeks solo, for the Women in AI Accelerator Spring 2026 Build Challenge. Initial commit went in at 7:06pm on May 9. I pushed to AWS Amplify . Build went green. I opened the live site, filled out the form, hit submit. Nothing. Twenty eight seconds later, "Request timed out." I told myself the bug was not in my code. Everything ran locally. This had to be a platform problem. That belief carried me all night. It mostly held up. The exception was the first thing I should have checked. Here is the commit log, because it tells the story better than I can: 19:06 Initial commit: Themis Lex MVP 20:31 refactor: migrate Bedrock auth to IAM compute role 22:29 diag: log credential env vars at runtime (booleans only, remove after fix) 22:40 fix: forward BEDROCK_MODEL_ID to SSR runtime via next.config.js env 22:50 fix: switch to InvokeModelWithResponseStreamCommand to beat 28s Lambda timeout ... 06:28 fix: remove unused type export that broke isolatedModules build 06:37 fix: end-to-end response streaming to beat Amplify 28s gateway timeout 06:48 fix: reduce max_tokens to 3000 to fit Amplify 30s timeout 06:52 fix: reduce max_tokens to 2000, 3000 still exceeded 30s timeout 07:00 fix: switch to Claude Haiku 4.5 to fit Amplify 30s timeout Ten and a half hours from first deploy to the fix that shipped it. That gap between 22:50 and 06:28 is me sleeping on it, which turned out to be the second most productive thing I did. The error message was the absence of an error message My f

Three bugs we found and fixed in our own pipeline this week Journeymen grades developer work against GitHub's server-side history. That only means something if the grading pipeline itself is reliable — so here's the honest engineering update, not the highlight reel. 1. Silent progress loss on connect-repo analysis runs A connect-repo analysis run could sit in processing status with no visibility into what stage it was actually at, or whether it had stalled. From a dev's dashboard, a slow run and a stuck run looked identical. We added explicit progress-stage tracking so a stuck run is visibly stuck, not silently pending. 2. A background worker timing out without a clear signal The Lambda-based worker handling asynchronous analysis jobs was hitting its timeout under certain repo sizes, and the failure mode wasn't obvious from the outside — a run would just never complete. We root-caused the timeout and fixed the underlying slow path. 3. Dead-letter queue with no observability Jobs that failed enough times to land in the SQS dead-letter queue were, until this week, invisible — no alerting, no in-product surfacing. We wired up observability so a DLQ arrival is now a visible signal instead of a silent dead end. Why post about our own bugs The entire pitch of Journeymen is "don't trust the self-reported version, trust the verified one." That standard has to apply to us too. All three issues: found, fixed, and shipped this week. journeymen.in

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry. ...