标签：#Engineering

In the previous article , I argued that AI isn't simply changing the tools we use to develop software, but shifting our work to a new level of abstraction. In this one, I want to address the problem that immediately follows: if we're going to write less and less code directly and agents are going to produce an increasingly larger part of it, how the hell do we know whether what they code is actually right? Because the answer obviously can't be “trust the AI, it's very smart”. Even though I personally develop code with AI today with practically no review, I don't blindly trust AI. Just as I don't blindly trust an engineer on my team. I don't even blindly trust myself. Blind trust is a security hole. And not blindly trusting someone doesn't mean distrusting them, it means having mechanisms to prevent their mistakes, or mine, from causing problems. That's why we've spent decades building mechanisms and methodologies around software development to detect, and avoid as much as possible, our mistakes. XP. Scrum. Tests. Code reviews. Pair Programming. CI. Static analysis. Permissions. Observability. Environments. Containers. Auditing... The question, therefore, shouldn't be whether we can trust an AI. The question should be what system do we need to build so we can use it without needing to blindly trust it? It's not deterministic One of the first objections is usually that if you ask it the same thing twice, it generates two different pieces of code. True. But if you give the same task to two different programmers, or to the same programmer with enough time in between, we'll very probably get two different implementations too, depending on the complexity of what we're asking. And if we've never required two developers to produce exactly the same code, why do we expect AI to produce exactly the same code from the same request? Isn't it enough for the result to satisfy the requested requirements? That it does what it's supposed to do. That it passes all kinds of tests. That

AI is disrupting career progression by eliminating the learning opportunities at each rung while simultaneously enabling people to perform above their experience level, Alasdair Allan explained in his talk Engineering Progression When AI Ate the Middle at QCon London. Fewer junior developers join the industry, and AI slows hiring at the entry level. By Ben Linders

A playable first-person shooter generated from one prompt would have sounded absurd not long ago. Now, videos of AI-built browser games that resemble Call of Duty and Counter-Strike are spreading across social media. On August 10, Axios reported on the rise of “one-shot” AI game prompting : give a model one detailed instruction, let it produce the code, and receive something you can play. This is a real milestone. It is also easy to misunderstand. A one-prompt game can prove that a model knows how to assemble controls, graphics, physics, enemies, and a recognizable game loop. It cannot prove that the result will stay interesting after the first few minutes. The first prompt creates the demo. The decisions after that create the game. Why These Demos Feel So Important Game ideas used to face a large gap between imagination and interaction. You could describe a mechanic, draw a map, or write a design document. But discovering whether the idea actually felt good required code, assets, an engine, and enough technical work to reach a playable build. Prompt-to-game tools are shrinking that gap. This change is not limited to experimental AI demos. Roblox recently announced mobile-first creation tools that turn text prompts into basic games , giving creators a starting point they can playtest, change, share, and publish. That starting point matters. A playable failure teaches you more than a beautiful design document. You can immediately discover that the movement is slow, the arena is empty, the objective is confusing, or the central mechanic is less interesting than it sounded. The value of one-shot generation is not that the first result is finished. It is that the first result arrives early enough to challenge your assumptions. A Recognizable Game Is Not Necessarily a Good Game A model can generate the visible parts of a familiar genre surprisingly well. Ask for a browser FPS and it may produce: First-person movement Weapons and ammunition Enemies that chase or shoot Hea

"Why not just use a cron job?" is the first question I get whenever someone sees an Airflow DAG. Fair question. Cron works. It's been around for decades. It's simple. The real answer isn't that cron is bad — it's that cron solves a different problem than Airflow does. Cron is a job scheduler . It runs a command at a fixed time. That's it. It doesn't know whether the command succeeded, whether its dependencies are satisfied, or whether it should even run at all today. It just fires the command and moves on. Airflow is a workflow orchestrator . It doesn't just schedule tasks — it models them as a graph of dependencies, tracks their state, retries failed ones, and gives you a UI to see what ran, what failed, and why. Here's where that difference actually matters. The problem cron can't solve Imagine a simple ETL pipeline: Extract raw data from an API Validate and clean it Load into a warehouse Run a transformation Send a Slack alert if anything fails With cron, you'd write five separate cron entries, one per step, and hope the timing works out. If step 2 fails but step 3 runs anyway, you now have bad data in your warehouse. If step 4 takes twice as long one day, you've silently broken your SLA. Nobody gets notified unless you manually add alerting logic to every script. With Airflow, you model this as a DAG: from airflow import DAG from airflow.operators.python import PythonOperator from datetime import datetime with DAG ( dag_id = " daily_etl " , schedule = " 0 6 * * * " , start_date = datetime ( 2026 , 1 , 1 ), catchup = False , ) as dag : extract = PythonOperator ( task_id = " extract " , python_callable = extract_data ) validate = PythonOperator ( task_id = " validate " , python_callable = validate_data ) load = PythonOperator ( task_id = " load " , python_callable = load_to_warehouse ) transform = PythonOperator ( task_id = " transform " , python_callable = run_transformation ) extract >> validate >> load >> transform Airflow guarantees the order. If validate fail

There are two ways to work with an AI agent and I had tried both. Write the thing yourself and hand over only the tedious parts. Or hand over the whole task and audit whatever comes back at the end. The first is slow. The second is fast right up until it is wrong, and by then the wrong thing is finished. I expected this series to be about forcing a third option into existence. Nine parts of making an agent follow a workflow it would rather skip. That is not what happened. I never had to enforce it once. The queue that started this had a payload contract nobody had verified, and each phase after that cost me something before it gave anything back. A plan that would not move until the risk register named the provider contract the brief had only guessed at. A build that missed nothing except what my own brief left out. A review that stopped handing back a feeling and started handing back a verdict on every requirement I had already called done. A matrix instead of a trusted green run. A rollback with a name on it before anything got called shipped. And a retrospective that would not let a lesson through until it had checked itself against the trail. Eight parts of that. What I did not expect was which part turned out to be automatic. The Fight I Expected Never Started By the time I finish writing a requirement, I already know roughly what it is going to cost. Most engineers do. You can feel the difference between a one-line fix and something that is going to touch four files and a migration before you have written a single line of it. What I assumed was that the agent could not feel that, and that policing the gap would be my job forever. Reminding it to run the chain. Catching it when it decided a spike was small enough to skip. It has not needed the reminder. Small bugs do not trigger a brief and a plan, and they should not. A standard requirement, a spike, anything long or cross-cutting, runs the full cycle in order. The classification lands where I would have put i

📺 Prefer to watch? 90-second YouTube Short · 💬 Telegram Originally published on software-engineer-blog.com . You run code concurrently all the time. But "concurrent" hides a critical choice: are you spawning separate processes or threads inside the same process? That choice decides whether one crash takes down your entire system or stays contained, and whether you're copying data between isolated worlds or racing to read the same memory. Mental model: A process is its own house; threads are roommates sharing one. Processes: Isolation at the Cost of Weight When you start a process, the operating system hands it its own private address space. That address space is walled off. Your process can't touch another process's memory—the OS enforces it at the CPU level. If your process crashes, it corrupts only its own memory. The kernel cleans it up. Every other process keeps running untouched. This is why browsers put each tab in its own process. One tab runs malicious JavaScript, spins into an infinite loop, or has a memory leak—that tab's process dies. The rest of your browser lives. You close the dead tab and open a new one. Your other tabs don't even hiccup. But isolation isn't free. Each process carries: Its own copy of the heap, stack, and memory pages Its own file descriptor table, open sockets, and kernel resources OS overhead to track and protect it Spawning a process is expensive—milliseconds on modern hardware, but measurably heavier than a thread. And if two processes need to share data, they can't just read the same memory. One process must copy data into a pipe or socket, send it across, and the other process must copy it out and into its own memory. That's overhead on every exchange. Threads: Speed and Sharing, With a Trap Threads live inside a single process and share that process's entire memory. The kernel doesn't wall them off from each other. When you spawn a thread, you're not duplicating the heap, the file descriptors, or the kernel state—you're just cr

I am testing my first automated end to end social media post automation system. which is created using the free tools. But it is very efficient and productive. i can use this thing in future posting on various platforms to tell people about my learning's and update about me. Tools : Make.com = I use this tool to mainly automate my system it include flow how things works and system is linked. Hashnode = I use this as a central blog and article publishing tool other tools is connected with it so content links is properly distributed. Google Ai Studio = I use this to integrate the ai in between this whole process which just do small job to add the engaging hook and the tags for the reach Buffer = I use to connect X (twitter) with this Because Make.com remove the platform X (twitter) to His integration. After the policy change of the platform. Dev.to = I use this to improve SEO of my post over the google search engine. Challenges : I cannot integrate the github actions with the hashnode becuase this feature is become paid on hashnode. May be in future i can do this thing using self written yml file, i am guessing Not sure will this 100 % work or not. Twitter integration as i described early that twitter integration is not present in the make.com so i use the another tool Buffer. The limits calculation, Their was a limits on each tools for their specific use case so i have to intentionally calculate them properly. Even the free tear of the twitter which is X is few hundreds words that's why i have to limit the text of the post, which is hook only, The threads creation i don't think it will be their in this tools which i am using, i will definitely find it if their. Solutions : Simply use other Way if this way is closed, use different tool for twitter May be in future i create yml file for the github actions but for now i am directly writing on hashnode. The dev.to does not provide feature of direct posting it save your cycle into draft so you have to manually click on pu

IBM and Red Hat have announced an expansion of Lightwell, introducing new commercial offerings designed to help organizations establish trusted, verifiable software supply chains for the age of AI-assisted software development. By Craig Risi

Meryem Arik discusses strategies for designing low-cost LLM inference architectures for high-volume, non-real-time workloads. She explains how software architects and engineering leaders can achieve order-of-magnitude cost reductions by making critical trade-offs across hardware, inference runtimes, speculative decoding, and smart queue reordering. By Meryem Arik

Originally published on Loop & Retry — field notes on building LLM agents that survive production. Here's the failure mode that surprises people who've only reasoned about agents statistically. You measure a per-step error rate — say 10% of steps produce something wrong — and you assume errors are independent, so a wrong step is a wrong step and the rest of the run is fine. Then you watch a real trajectory and see something else: step 4 gets a fact slightly wrong, step 5 reasons on top of that wrong fact and commits harder, step 6 takes an action premised on both, and by step 8 the agent is confidently executing a plan that was doomed at step 4. One mistake became five. The errors weren't independent — they were coupled through the context , and coupling is what turns a 10% step-error rate into a run that's wrong far more than 10% of the time. This is the cascade : a single fault amplifying down a single trajectory. It's distinct from the failure I wrote about in distributed retry patterns , where the problem is one bad condition hitting many workers at once — that's a blast radius, a horizontal spread. The cascade is vertical: it spreads through time within one run, because an agent's own past output is its future input. This post is about the vertical kind, why it's structural rather than bad luck, and where you can cut it. Why coupling is the default, not the exception A stateless function that fails just returns an error. An agent that fails does something worse: it writes the failure down where it can read it again. The mechanism is the same one that makes agents work at all — the transcript accumulates, and every step conditions on everything before it. That's a feature for carrying intent forward. It's also the exact channel a mistake travels down. Three ways a single fault propagates through the context: Poisoned premise. The agent derives or retrieves a wrong fact — a misparsed tool result, a hallucinated ID, a stale value — and it lands in the transcript a

JetBrains has described how it began centralising AI usage after development-related spending increased roughly tenfold in six months. Rather than restricting engineers to a small set of approved tools, the company built a shared access and accounting layer intended to preserve tool choice while giving teams greater visibility and control over consumption. By Matt Foster

Craft externalizes nothing and transfers by apprenticeship. Engineering writes the governing relation down where someone else can find it wrong. Four times now I have written the same three sentences in different notations, for four problems that looked unrelated: a design method, a coding technology, an architecture-derivation procedure, and a contract-modelling tool. I noticed the repetition only after the fourth. Here it is, stated as precisely as I can manage. Structure is derived from the attribution of forced change. The attribution is kept as an explicit, checkable artifact. The derivation refuses rather than guesses when its inputs underdetermine the answer. Three clauses, each carrying weight. Drop one and look at what remains. Drop derived and you have a documentation exercise: the structure was chosen first and the attribution written to match. This is the normal case, and it makes no prediction, so nothing can disagree with it. Drop explicit artifact and you have taste. Real, valuable, and transferable only by apprenticeship. Drop refusal and you have a generator that answers every question. Its answers carry no information, because it was always going to produce one. The third clause has a lineage worth claiming Type inference has refused for fifty years. Hindley-Milner unification fails rather than picking a plausible substitution: when two types cannot be reconciled, the answer is an error, not a guess. Core HM needs no annotations at all -- it infers principal types, and that is the point. The interesting part is what happened when later extensions broke that guarantee. Type classes admit programs whose type is inferable while the instance to use is not; GADTs and polymorphic recursion break principality outright. In each case the compilers were free to pick a plausible candidate, and they demand an annotation instead. Build systems joined later: Bazel refuses an undeclared dependency rather than resolving it from ambient state ( this rule is missing

Enterprise Java developers have a new superpower—drive GitHub Copilot from idiomatic Java code with annotations, virtual threads, and more. The post Using the GitHub Copilot SDK for Java appeared first on The GitHub Blog .

Three semantic layers, three architectures, three very different bills. All three will define what a metric means. None of them proves an AI agent is allowed to run it. Quick orientation dbt Semantic Layer Cube AtScale Core idea Metrics as version-controlled code Headless API in front of metrics OLAP-style aggregate acceleration Strongest when You want engineering discipline Many apps consume the same numbers Heavy, stable aggregate workloads Modelling Hand-authored YAML Hand-authored data model Hand-authored cubes Cost driver Plan tier + query volume Pre-aggregation builds + compute Quote-based licence + compute Governance Upstream, in the warehouse In front of the API On the cube Each is competent at what it was built for. If your consumers are dashboards and analysts, any of the three will serve you. The question none of them answers An agent doesn't arrive with a metric name. It arrives with an intent in English and has to work out which entities, which grain, which joins, and whether it's entitled to any of it. That exposes two gaps every one of these shares: Undefined intent has no answer. Coverage is whatever someone remembered to model. Business questions don't respect that boundary. Authorisation is checked around the query, not inside it. A filter applied after execution means the data already moved. What to actually evaluate on Ignore feature matrices and score these five: Answer a question nobody modelled, on your schema Show why one join path was chosen over two others Same question, two users with different entitlements — show both SQL statements Ask something ambiguous. Refusal or guess? Reproduce a number from six months ago with the definitions then in force Most evaluations stop at 1. Numbers 3 and 5 are the ones that decide whether the thing ships in a regulated business. The full breakdown — architecture-by-architecture comparison, cost profiles, and the migration implications of each — is here: 👉 dbt Semantic Layer vs Cube vs AtScale: Choosing a

A compiled application cannot hot-reload itself if its main loop, window context, and memory allocations live inside the binary being recompiled. The application must be split into two layers:Host Shell (Stable Execution Root):Statically compiled once.Manages the OS window, render loop, event polling, network sockets, and high-level heap allocations.Exposes a dynamic symbol loader (dlopen / LoadLibrary or a dynamic WebAssembly runtime execution context).Guest Module (Hot-Swappable Logic):Compiled as a shared dynamic library (.so, .dylib, .dll) or an isolated WebAssembly (.wasm) module.Contains frame updates, business rules, rendering instructions, and component tree logic.Exports explicit interface hooks (init, update, render, pre_reload, post_reload).The Hot-Reload PipelineWhen a developer edits source code in a compiled language (e.g., modifying a Rust UI render function or a C# algorithm), the dev server orchestrates a zero-downtime swap through this explicit pipeline:1.File Watcher & Fast Incremental Compile:Sub-second artifact generation.The watcher detects source changes and invokes an incremental compilation pass using dynamic linking configurations (e.g., -rdynamic, dynamic C-runtime links, or fast lld/mold linkers) to output a versioned binary artifact (logic_v2.so).2.Live Manifest Update:Atomic state & symbol mapping emit.The dev server emits an updated JSON manifest containing module hash, exposed symbol tables, binary payload locations, and updated asset hashes over a WebSocket/IPC stream to the Host Shell.3.State Snapshot & Freeze:Preserving user context.The Host Shell signals pre_reload() to the currently loaded logic_v1.so. The guest logic serializes volatile runtime state into a host-managed memory buffer or leaves pointers active inside a host arena.4.Dynamic Unload & Library Swap:Operating system symbol rotation.The Host Shell unloads logic_v1.so (releasing file locks via temporary copy paths on OS platforms like Windows), loads logic_v2.so, and re

Kennedy Torkura discusses practical GenAI red teaming techniques to safeguard LLMs and knowledge bases against security threats like data poisoning and LLMjacking on AWS. He explains how engineering leaders and architects can bridge traditional cloud security with MITRE ATLAS frameworks to proactively identify vulnerabilities, implement guardrails, and secure production AI applications. By Kennedy Torkura

Cloud Native Buildpacks, which graduated within the CNCF in July 2026, move base image choice out of per-service Dockerfiles into a single builder owned by platform engineering, enabling fleet-wide patching. BellSoft's hardened Paketo builder is the latest sign that vendors now treat the builder, not the Dockerfile, as the container security control point. By Mark Silvester

When building and validating US stock quantitative strategies, I used to focus solely on core market data metrics. Like most individual quantitative developers, I prioritized the integrity of price candlesticks and trading volume data, assuming that complete K-line datasets would guarantee reliable backtesting outcomes that align with real-market performance. This assumption held true for small-scale tests and short-cycle verification, until I encountered persistent inconsistencies between historical backtest reports and live trading results. After thorough troubleshooting of strategy logic, parameter settings, and sliding point simulation, I finally pinpointed the root cause — inconsistent and inaccurate timestamp processing from market data APIs, a trivial-looking but critical engineering detail that most developers overlook. Most engineering teams devote massive effort to verifying the accuracy of US stock API quote data, yet ignore standardized processing for time fields. In quantitative trading systems, timestamp offset and timezone disorder are far more impactful than superficial chart display errors. They directly distort candlestick combinations, disrupt technical indicator calculations, and ultimately mislead the entry and exit signal judgments of trading strategies. Core Requirement: Time-series consistency for valid backtesting Market data is essentially a continuous time-series stream, where price and volume merely represent transaction outcomes at specific timestamps. The time dimension acts as the fundamental anchor that defines the exact position of every single trade in the market timeline. Unlike A-share market data that adopts a unified time standard, US stock data providers deliver multiple incompatible time formats across different APIs, including pure UTC time, US Eastern trading time, and original exchange timestamp fields. Without unified parsing and conversion logic in your program, timestamp misalignment and data dislocation are inevitable.

You open a new Spring Boot project and you create a DTO, then an entity, and you’re staring at getters, setters, constructors, equals() , hashCode() , toString() . Someone on the team suggests to put lombok’s @Data , or “just slap @Builder on it, it’ll be cleaner.” Forty lines become five and it looks great in the PR. Then it hits a real codebase, OpenAPI generation doesn't behave the way the build expects. Hibernate meets an auto-generated equals() and gets confused about identity. Something throws through a generated builder hierarchy at 2 AM, and the method you need to inspect doesn't exist in any file you can open. Eleven years into enterprise Java, my rule is simple: Lombok doesn't touch core application behavior. Not because writing a getter is interesting - it isn't, but because the handful of lines it saves rarely covers the compiler magic, tooling friction, and debugging problems it adds to something that has to survive for years after you've moved on to another project. "It just removes boilerplate" Worth asking what's actually being removed, though. A getter is part of your public API. A setter is a mutation point someone decided to expose. A constructor defines what states an object is allowed to enter. equals() and hashCode() define identity. toString() is what shows up in your logs when things go wrong at 3 AM. Write those by hand and they live in the source - visible, searchable, debuggable, owned by whoever's reading the file. Generate them with Lombok and the behavior is still there, it's just moved somewhere you can't see it without a separate step. The annotation most people reach for first time is @Data : @Data @Entity public class CustomerEntity { @Id @GeneratedValue private Long id ; private String email ; @OneToMany ( mappedBy = "customer" ) private List < OrderEntity > orders ; } One line and you get getters, setters, toString() , equals() , hashCode() across every field. For a JPA entity that's already a problem before you've written any bus

I thought this would be a simple task. I already had a Python Kafka producer running. Kafka was up in Docker. The topic existed, and I could send a message into it successfully. The next step sounded straightforward: Python Producer ↓ Kafka ↓ Spark Structured Streaming All I wanted Spark to do was read a JSON message from a Kafka topic. Instead, I ran into one error after another. At first, it looked like one problem: Spark cannot read Kafka. It was not one problem. It turned into a chain of failures across several different layers: Python / PySpark ↓ Spark runtime ↓ Kafka connector ↓ Hadoop / Windows ↓ Docker ↓ Kafka networking ↓ Ivy dependency resolution The useful part of this experience was not any single fix. It was learning how to separate the layers and stop treating every error as a problem in my Python code. This is the full debugging path. What I Was Building This was part of an financial data engineering project. The batch side of the project already looked roughly like this: Financial Data Source ↓ Python ingestion ↓ AWS S3 ↓ Snowflake ↓ dbt ↓ Financial anomaly models I wanted to add a streaming extension for newly arriving financial events. For the first version, I kept it intentionally simple: Python Kafka Producer ↓ Kafka topic: financial_events ↓ Spark Structured Streaming The producer sent a simulated financial event: { "company_id" : "COMPANY_001" , "company_name" : "Sample Company" , "report_type" : "quarterly_report" , "reporting_date" : "2026-08-08" , "event_id" : "FIN-20260808-001" , "source" : "simulated_financial_event" } Kafka accepted the message successfully. I could even read it with Kafka's console consumer. So Kafka itself was working. Then Spark entered the picture. Failure #1: PySpark Worked, but spark-submit Didn't I installed PySpark: pip install pyspark Then I installed Java 17 and verified it: java -version After reopening my terminal, Java was available. I tested Spark directly through Python: python -c "from pyspark.sql import S