标签：#Ring

I started my career in the late 2010s, and I have had a front-row seat to the growth of the industry that has given me everything: software engineering. Looking back over the last decade, I have mixed feelings about some of the calls I made. And I am seeing the same patterns play out again now. So for engineers who are confused about where this is headed and how to navigate it, here is how I think about it. Generalist SWEs were a product of cheap money The late 2010s, I saw an huge amount of startup funding, globally. Flipkart, Snapdeal, Jugnoo, and hundreds of others were scaling hard and one hiring pattern I saw was that: everyone wanted generalist software engineers. People who could easily get upto speed across the stack.- backend, frontend, infra, deployment and simply ship. Building software was expensive. Automation was still low. Kubernetes had just gone mainstream. Shipping still meant a surprising amount of manual work: SSH-ing into servers, copying artifacts around, running mvn builds by hand, debugging deployments straight in production, duct-taping infrastructure that today you would never touch. Companies fought over engineers who maximized feature throughput. Breadth was a premium, because every extra engineer increased the rate at which software got built. It helped because the money was also free and VCs rewarded growth over efficiency, and hiring software engineers in bulk was the easiest way to spend it. Pull up a resume from an engineer who started around that time and you will usually see the same shape: a long list of technologies and frameworks, broad and adaptable, but rarely deep in any one thing. There was no incentive to go deep. LLMs Changed The Dynamics LLMs did not kill software engineering. It compressed the cost of implementation. The work that got hit first was the work that was already standardized: CRUD apps; API integration and glue code; Framework-heavy backend work; Frontend scaffolding; Standard architectural patterns. What use

As developers, data engineers, or analysts, we’ve all been there: you download a massive database export, a logging stack dump, or a transaction archive, only to find it's a multi-gigabyte JSON file. You try to import it into a spreadsheet or run it through a standard online converter, and boom—your browser tab freezes, crashes, or shows the dreaded "Out of Memory" screen. Even worse, if you try to use standard cloud-based online tools, you might have to wait for a 500MB upload to complete, only to hit a rigid file-size cap or, worse, compromise sensitive data privacy by uploading corporate logs or database records to a third-party server. In this guide, we will explore: Why large JSON files crash standard parsers (the V8 heap limit problem). How streaming architectures solve this by reading data chunk-by-chunk. NDJSON (JSON Lines) vs. JSON Arrays and how to stream them. A browser-native, 100% offline tool to convert large JSON to CSV instantly: Parsify's Large JSON Stream Converter . How to implement your own basic browser-based JSON streaming parser in JavaScript. 1. The Anatomy of a Memory Crash (Why JSON.parse Fails) If you are using JavaScript or Node.js, the simplest way to read and parse a JSON file is to load the file into memory and run JSON.parse(). const fs = require ( ' fs ' ); // Naive approach: Will crash on a 1GB+ file fs . readFile ( ' database-dump.json ' , ' utf8 ' , ( err , data ) => { if ( err ) throw err ; // POINT OF FAILURE: V8 Heap Out of Memory const records = JSON . parse ( data ); records . forEach ( record => { // Process record... }); }); This works fine for small config files. But once your JSON file reaches 100MB, 500MB, or 1GB+, this approach is guaranteed to trigger a fatal crash: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory Why does this happen? The String Duplication Overhead: When you load a 1GB file into memory, you first allocate ~1GB of RAM for the raw text string. The

Over the past weeks, I’ve been sharing a series of posts that gravitate around one question: How do...

Every few months, I see another post declaring: "Prompt engineering is dead." Usually, the argument goes something like this: AI models are getting smarter. They understand natural language better. You no longer need carefully crafted prompts. On the surface, that sounds reasonable. But after building AI workflows and experimenting with modern frameworks, I think the opposite is happening. Prompt engineering isn't disappearing. It's evolving. And if you're building AI applications, not just chatting with AI, you'll probably rely on it more than ever. Prompt Engineering Was Never About Fancy Prompts One of the biggest misconceptions is that prompt engineering is about writing magical sentences that somehow unlock hidden AI capabilities. It isn't. Good prompt engineering is about giving an AI system exactly what it needs to complete a task reliably. Consider these two examples. Poor prompt: Write Python code. Better prompt: Write a Python FastAPI endpoint that accepts a CSV upload. Requirements: Use Python 3.12 Validate file type Handle exceptions Return JSON responses Include comments explaining each step The second prompt isn't "clever." It's simply clearer. And clarity scales. AI Models Are Better, But They Still Need Context Modern LLMs have become incredibly capable. They can: Generate code Explain algorithms Debug applications Write tests Refactor functions But they still don't know: Your architecture Your coding standards Your API contracts Your deployment strategy Your business requirements That information comes from you. And the way you provide it matters. Prompt engineering is fundamentally the practice of supplying useful context. Every AI Framework Depends on Good Prompts Take a look at the most popular AI frameworks. Whether you're using: LangChain LangGraph CrewAI LlamaIndex Every one of them eventually sends prompts to an LLM. Even sophisticated agent systems are built from sequences of prompts. Agents don't eliminate prompt engineering. They multiply

Apache Iceberg looked like the answer to everything when we first adopted it. Open format, ACID transactions, time travel, schema evolution. We migrated our Hive tables, ran a few queries, and felt good about life. Three months later, our S3 costs doubled. Queries that used to take 10 seconds were taking 4 minutes. Metadata operations were timing out. Nobody on the team could explain why. That was the beginning of a real education in how Iceberg actually behaves in production. This post covers what I wish someone had told us before we went all-in. The Small Files Problem Is Not Optional Iceberg is append-friendly by design. Every micro-batch write, every streaming insert, every incremental load creates new Parquet files. Each file also gets its own metadata entry. After a week of hourly loads, you might have 10,000 files in a single partition where you wanted 20. The result: Iceberg's metadata layer has to plan queries across thousands of file manifests. Planning takes longer than execution. Your 10-second query becomes a 4-minute query, and your users start filing tickets. Fix: automate compaction from day one. In Spark, compaction is called rewrite_data_files . The basic call looks like this: -- Run this on a schedule, not on-demand CALL iceberg_catalog . system . rewrite_data_files ( table => 'analytics.events' , strategy => 'binpack' , options => map ( 'target-file-size-bytes' , '134217728' , -- 128MB target per file 'min-input-files' , '5' -- only compact if 5+ small files exist ) ) Target file size of 128MB to 512MB is the practical sweet spot. Smaller than that, you still have too many files. Larger, and your query engines cannot parallelize reads efficiently. If you are not using Spark, PyIceberg exposes compaction through the table maintenance API (as of 0.7.x). For Flink or Trino-only shops, schedule compaction as a separate Spark job. Yes, it is annoying, but it is the right call. Hidden Partitioning Is the Feature You Are Probably Ignoring Old Hive parti

While AI dominates the layoff narrative, engineers as a share of total new hires have actually increased, according to SignalFire data.

Your domain event fires. Your notification service queries the DB for the entity that just got saved. It finds nothing. You add a log line. It starts working. You remove the log. It breaks again. That's not a race condition. That's @EventListener . What's actually happening Spring's @EventListener fires synchronously, inside the calling thread, before the transaction commits. The DB row exists in Hibernate's session — but it hasn't been flushed and committed yet. Other connections, including the one your listener opens when it calls findById , can't see it. The log statement "fixes" it because the delay gives Hibernate time to flush. Remove the log, the flush doesn't happen in time, and you're back to an empty Optional . Here's the broken setup: @Component public class OrderEventListener { @EventListener // fires MID-TRANSACTION, before commit public void onOrderCreated ( OrderCreatedEvent event ) { // Transaction not committed yet. // Other DB connections see nothing. Order order = orderRepository . findById ( event . getOrderId ()) . orElseThrow (); // ← throws here, row doesn't exist yet notificationService . notifyCustomer ( order ); } } The obvious fix and what it costs you Spring ships @TransactionalEventListener for exactly this. Set phase = TransactionPhase.AFTER_COMMIT and the listener fires after the transaction commits. The row is visible. findById returns the order. Problem solved. @Component public class OrderEventListener { @TransactionalEventListener ( phase = TransactionPhase . AFTER_COMMIT ) public void onOrderCreated ( OrderCreatedEvent event ) { // Transaction committed. All connections see the row. Order order = orderRepository . findById ( event . getOrderId ()) . orElseThrow (); // ← works fine notificationService . notifyCustomer ( order ); } } But the trade-off is real. Your listener is now decoupled from the transaction. If the listener fails — notification service is down, the email throws, the external API times out — the transaction alrea

Technology companies are extending AI beyond code generation into earlier stages of the software lifecycle, including PRD validation, design inputs, and code review. Initiatives from Uber, DoorDash, and Cloudflare highlight a shift toward AI-driven governance layers that evaluate engineering artifacts before implementation while preserving human oversight across the development pipeline. By Leela Kumili

AI has made it easier to produce working software. That part is real. It can write code, draft documents, research a topic, scaffold a prototype, and debug a problem faster than most teams can finish writing a decent ticket. But faster building doesn't automatically mean better product decisions. That's the part I keep coming back to. For decades, software teams optimized around delivery. Requirements, design, development, QA, release. Waterfall softened into Agile. Agile grew into DevOps. The practices changed, but the assumption underneath stayed pretty stable: building software is expensive, so plan carefully before you start. That made sense because, for a long time, it was true. Now that assumption is breaking. AI is doing to software what calculators did to accounting. It isn't eliminating the job. It's moving the job up a level. The syntax, boilerplate, first draft, and some of the debugging are getting offloaded. The work doesn't disappear. The bottleneck moves. Learning is still expensive Here's what didn't get cheaper: understanding what people actually need getting stakeholders aligned deciding what evidence would change your mind putting something real in front of users reading the signal without fooling yourself The old question was: Can we build it fast enough? The new question is: Do we understand the problem well enough? That sounds like a small shift, but it changes the work. It changes what strong engineers spend time on. It changes what product people need from engineering. It changes how teams should define "done." If the code ships but nobody learns anything, did the team actually move forward? Sometimes yes. Often no. Users don't know until they can touch it People are not great at specifying requirements up front. Not because they're difficult. Because they're human. Most of us don't know how we feel about something until we can react to a version of it. A mockup. A prototype. A rough slice. A real workflow with sharp edges. So the fastest pat

Naomi Saphra discusses 5 rules governing language model behavior, breaking down why LLMs act like populations rather than individuals. She explains how tokenization creates strange semantic blind spots and highlights the mechanics of sycophancy, showing how models leverage subtle data associations to match user biases and demographics - even guessing political views based on favorite sports teams. By Naomi Saphra

Model Context Protocol (MCP) is an open standard for connecting AI apps to tools and data sources. A useful way to think about it is as a USB-C port for AI: one standard interface that lets different models plug into different capabilities without custom glue code for every integration. In this project, we combine MCP, Spring AI, and Google Gemini to build a chat app that can answer weather questions using real tools instead of hallucinating. The system has three parts: MCP tool server - a Spring Boot service that exposes weather and geocoding tools AI chat agent - a Spring Boot service that uses Spring AI + Gemini and calls MCP tools when needed React chat UI - a lightweight frontend for sending messages and rendering replies The result is a small but realistic architecture you can extend into a production assistant. Architecture User (Browser:3000) | POST /api/chat v AI Agent (Spring:7171) -- MCP / Streamable HTTP --> MCP Server (Spring:7170) | | | Google Gemini | Bright Sky API (weather) | | OpenStreetMap Nominatim (geocoding) v v Chat response Tool execution The full source code is available on GitHub . 1. The MCP Tool Server The tool server is a Spring Boot application that exposes MCP tools through Spring AI's annotation scanner. It runs on port 7170 and uses Streamable HTTP for transport. Dependencies <dependency> <groupId> org.springframework.ai </groupId> <artifactId> spring-ai-starter-mcp-server-webmvc </artifactId> </dependency> <dependency> <groupId> org.springframework.boot </groupId> <artifactId> spring-boot-starter-web </artifactId> </dependency> Defining tools With Spring AI, a tool is just a Spring bean method annotated with @McpTool : @Component public class WeatherTool { private final WeatherToolService weatherToolService ; public WeatherTool ( WeatherToolService weatherToolService ) { this . weatherToolService = weatherToolService ; } @McpTool ( name = "get_current_weather" , description = "Get current weather by dwd_station_id or by lat/lon" ) p

As soon as August, a giant silver bullet will cut its way through the dry air of the southwestern US and cross the Pacific to reach the coast of Japan. Once there, the roughly 200-foot-long craft, built by the New Mexico–based company Sceye, will park some 18 kilometers above the ocean’s surface, in a wispy-thin…

Introduction When a query runs in ClickHouse®, the database does much more than simply read data and return results. Before execution begins, ClickHouse® parses the SQL statement, analyzes it, applies optimizations, and builds an execution plan that determines the most efficient way to process the query. Understanding query execution plans is one of the most valuable skills for anyone working with ClickHouse®. They provide visibility into how queries are executed, helping you identify bottlenecks, validate optimization efforts, and troubleshoot performance issues. In this article, we'll explore how ClickHouse® generates execution plans, the different EXPLAIN modes, and how to interpret them for better query optimization. Why Query Execution Plans Matter A SQL query defines what data you want, but it doesn't explain how the database retrieves it. Consider the following query: SELECT country , count () FROM events GROUP BY country ; Although the query looks simple, ClickHouse® must determine: Which data parts to read Whether primary indexes can reduce the scan If data skipping indexes can be used How aggregation should be performed Whether parallel execution is possible How intermediate results should be merged A query execution plan provides answers to these questions, making it an essential tool for performance tuning. The ClickHouse Query Lifecycle Every query passes through several stages before producing results. The lifecycle typically looks like this: SQL Query │ ▼ Parser │ ▼ Analyzer │ ▼ Optimizer │ ▼ Query Plan │ ▼ Execution Pipeline │ ▼ Results Each stage plays an important role: Parser validates SQL syntax. Analyzer resolves tables, columns, and expressions. Optimizer applies query optimizations. Query Plan determines the logical execution steps. Pipeline distributes work across multiple threads. Execution processes the data and returns the results. Understanding this workflow makes execution plans much easier to interpret. Introducing the EXPLAIN Statement

I got a take-home assignment last year from a company I was genuinely excited about. "Should take about four hours," the recruiter said. Build an ingestion pipeline, model the data, write tests, document your design decisions, and prepare a 15-minute presentation walkthrough for the panel. Four hours. I laughed, closed my laptop, and started on it the next morning like it was a sprint. Sixteen hours later I had something I was proud of. Clean pipeline, solid tests, real documentation. I submitted it on a Sunday night. Monday I got a form rejection. No notes. No feedback. Not even which stage I failed. Just "we've decided to move forward with other candidates" and a link to their Glassdoor page. That was the moment I stopped pretending take-homes are assessments. They're consulting gigs. Unpaid ones. The Scope Creep Nobody Talks About Five years ago, a data engineering take-home was a focused exercise. Model this dataset into a star schema. Write a few SQL transforms. Maybe a short README. Two to four hours, tops. Bounded, reasonable, and actually useful for evaluating how someone thinks about data. That version is dead. Today, 68% of companies use take-home tests, up 12% year over year. And the scope has quietly ballooned into something unrecognizable. Full pipeline implementations. Test suites with coverage thresholds. Documentation that reads like a design doc. A presentation follow-up where you defend your architecture to a panel. We're talking 10 to 20 hours of work, routinely, for a role you haven't been offered. Industry best practice caps take-homes at 90 minutes of expected effort. The reality? Candidates consistently take 2x longer than company estimates to reach submission quality. That "four-hour" assignment is an eight-hour assignment. That "weekend project" is a week of evenings. And 25% of companies are still handing these out like they're reasonable asks. Here's the part that makes my eye twitch: 71% of engineering leaders openly say take-homes no lon

Zabbix is an open-source monitoring platform that tracks the health and performance of servers, networks, applications, and services, with built-in alerting and visualisation. This guide deploys the Zabbix server, web UI, agent, and MySQL database using Docker Compose, with Traefik handling automatic HTTPS for the dashboard. By the end, you'll have Zabbix monitoring its own host with a secured dashboard at your domain. Set Up the Project Directory 1. Create the project directory: $ mkdir ~/zabbix-docker $ cd ~/zabbix-docker 2. Create the environment file: $ nano .env DOMAIN = zabbix.example.com LETSENCRYPT_EMAIL = admin@example.com MYSQL_PASSWORD = YOUR_DB_PASSWORD MYSQL_ROOT_PASSWORD = YOUR_ROOT_PASSWORD Deploy with Docker Compose 1. Add your user to the Docker group: $ sudo usermod -aG docker $USER $ newgrp docker 2. Create the Compose manifest: $ nano docker-compose.yaml services : traefik : image : traefik:v3.6 container_name : traefik restart : unless-stopped command : - " --providers.docker=true" - " --providers.docker.exposedbydefault=false" - " --entrypoints.web.address=:80" - " --entrypoints.websecure.address=:443" - " --entrypoints.web.http.redirections.entrypoint.to=websecure" - " --entrypoints.web.http.redirections.entrypoint.scheme=https" - " --certificatesresolvers.le.acme.httpchallenge=true" - " --certificatesresolvers.le.acme.httpchallenge.entrypoint=web" - " --certificatesresolvers.le.acme.email=${LETSENCRYPT_EMAIL}" - " --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json" ports : - " 80:80" - " 443:443" volumes : - /var/run/docker.sock:/var/run/docker.sock:ro - ./letsencrypt:/letsencrypt networks : - zabbix-net mysql-server : image : mysql:8.4.8 container_name : zabbix-mysql environment : MYSQL_DATABASE : zabbix MYSQL_USER : zabbix MYSQL_PASSWORD : ${MYSQL_PASSWORD} MYSQL_ROOT_PASSWORD : ${MYSQL_ROOT_PASSWORD} volumes : - ./mysql-data:/var/lib/mysql networks : - zabbix-net restart : unless-stopped zabbix-server : image : zabbix/zabbix-se

Your prompts are fine. Your AI output is still garbage. You write carefully. You're specific. You ask for the format, the tone, the length. Hit enter. The AI responds with something that sounds like it was written by a committee of lawyers having a really bad day. Here's what you don't realize: You're not telling the AI to do something. You're describing the problem, and the AI is solving for the statistical average. The fix isn't more detailed instructions. It's three examples. That's it. Three. Not ten, not one, three. This post is the complete guide to few-shot prompting — the single highest-leverage move in prompt engineering. By the end, you'll have a template you can copy into any AI and watch your output quality jump 5x. Prefer watching? Here's the 3-minute version Otherwise, read on — everything's below. Why Instructions Fail (And Examples Work) When you tell an AI to "be funny," it's working off a fuzzy statistical average of everything labeled "funny" in its training data. When you show an AI what you think is funny, you're giving it a precise pattern to match. Here's the difference: ❌ Instruction: "Write a funny one-sentence movie summary" Result: A lukewarm joke that lands in the middle of the comedy bell curve. ✅ Pattern: Funny summary of The Lion King: Cub loses dad. Cub becomes king. Funny summary of Finding Nemo: Dad fish swims very far for his son. Funny summary of Titanic: [AI fills this in] Result: Boy meets girl. Boat meets iceberg. Oops. Same AI. Different universe. The only thing that changed: you showed it the pattern instead of describing it. The Science (Why This Isn't Magic) Language models predict the next token by pattern matching. They've seen millions of prompt-response pairs and learned: "When a prompt looks like this , the output usually looks like that ." One example could be a fluke. Two examples might be a coincidence. Three examples are clearly a pattern. The AI recognizes the pattern and completes it. This is exactly how humans l

I've been working as Software Developer mainly focussed on Java and builts many application using Eclipse RCP framework or VS Code Application. Almost all the time I had to deal with multiple large files (either read/generate/validate) them which seemed very difficult and some of them almost impossible as most of them would be dependant on each other and would be referencing each other (just like how java files work together). Now assume client1 requires the same content in multiple Json files and client2 needs it in xml files. We couldn't go on writing a different application or go on adding if conditions and blah blah blah !!!! Wouldn't it be easier if as soon as I execute the application it generates the content in whatever format I choose and also taking care of dependencies/ references (like adding import statements). Additionally integrate with features of IDE and provide proposals, perform validations on the fly. Rela World Examples : Try googling Arxml once (Trust me I've dealing with these files for almost 7 years and it's always a nightmare to debug these) Solution: Xtext framework In this tutorial, I will show you how to use Eclipse Xtext and Xtend to build a simple, readable DSL that automatically generates Java boilerplate for you. Fair Warning: There will be no running executions screenshots or anything. You are gonna have to run it yourself and check the results and of course questions are always welcome in the comments section. But if for some reason you are unable to replicate this then let me know I'll try to explain further. I believe the best way to learn is by doing it yourself. The Goal: What are we building? Instead of writing 100 lines of Java with private fields, getters, and setters, we want our developers to write 5 lines of code in our own custom language (basically you can create your own programming language with your own custom syntax), like this: entity User { var name : String var age : Integer } When this file (assume file extension

If you've ever tried to set up Prometheus by following the official getting-started path, you're likely to find a path that does not follow your infrastructure model. Out of the gate, page one mentions kube-prometheus-stack. Page two wants you to install a Helm chart, and page three assumes you already have a cluster running. The documentation for monitoring plain Linux servers is in there somewhere, but you have to dig for it. When you do find it, the tone suggests you are doing something slightly old-fashioned. If that sounds like your setup, the tooling is making this harder than it actually is. Monitoring a fleet of Linux VMs is fairly simple and has been for years. It is just obscured behind documentation that would prefer to sell you something bigger. Modern infrastructure tooling has quietly decided everyone runs Kubernetes. If you don't, the assumption is that you eventually will. Meanwhile, most real-world infrastructure still runs on VMs. TL;DR: Modern observability documentation often assumes you're running Kubernetes. Most small teams aren't. If you're managing a fleet of Linux VMs, node_exporter plus Prometheus gives you everything you need for infrastructure monitoring with a single lightweight agent and a straightforward deployment model. No cluster required. VMs are often the answer For most small businesses, running VMs instead of Kubernetes does not mean you failed to evolve. Most workloads under a certain scale perform better on VMs: One process per box, predictable resource limits, and the ability to ssh in and look at what's happening, which makes it easier to keep track of the infrastructure as a whole. They're cheaper, both financially and in the mental overhead of running them. Backups and snapshots are straightforward in a way stateful Kubernetes still isn't. There's no control plane that itself needs monitoring and upgrades and care. Kubernetes solves problems that mostly pertain to companies with dozens of engineers and hundreds of service

Deploying a Spring Boot backend should be simple in theory. Build the JAR, set the environment variables, connect the database, and ship it. In practice, my deployment exposed several assumptions that worked locally but failed immediately in the cloud. I recently deployed a modular Spring Boot application to Render using Docker, Render Blueprint, PostgreSQL, Redis, Flyway migrations, Spring profiles, Hibernate/JPA, and environment variables. The application worked locally with MySQL and Redis, but deployment exposed several production-specific issues that were easy to miss in local development. This article documents the problems, why they happened, and how I fixed them properly. Who This Article Is For This article is useful if you are deploying a Spring Boot application to Render and your local setup uses MySQL, Redis, Flyway, Docker, or a multi-module Maven structure. It is especially relevant if you are moving from a local MySQL setup to PostgreSQL in the cloud. The Stack The backend was a Java 17 Spring Boot application with multiple Maven modules: alagbafo/ ├── api-contracts ├── core ├── users ├── orders ├── payments ├── wallet ├── notifications ├── admin ├── subscriptions └── app The app module was the actual Spring Boot entry point. Locally, the project used MySQL and Redis: spring.datasource.url = jdbc:mysql://localhost:3306/alagbafo spring.datasource.driver-class-name = com.mysql.cj.jdbc.Driver spring.data.redis.host = localhost spring.data.redis.port = 6379 For Render, the target setup was: Spring Boot app PostgreSQL database Redis-compatible Key Value store Docker deployment Flyway migrations Render Blueprint was the best fit because it allowed the infrastructure to be described in a render.yaml file. Step 1: Dockerfile for a Multi-Module Spring Boot App Because the project was a multi-module Maven application, the Dockerfile had to copy all module pom.xml files before copying the source code. This improves Docker layer caching because dependencies can b

Payment data pipelines fail in ways that ruin a payments engineer’s week, and the failures rhyme. The dashboards froze. Fraud scores arrived after the transaction had already cleared. Settlement reports came in stale. Nobody slept. The frustrating part is that the same data architecture had run fine for years. So, what changed? The honest answer is that batch thinking does not survive contact with real-time payments. A lot of banks built their data foundations in an era when nightly jobs were good enough. Load the warehouse overnight, run the reports in the morning, move on. That rhythm worked when money moved slowly. It does not work when a customer expects an instant confirmation and a fraud engine has milliseconds to make a call. Here is where things crack. Real-time payment rails push a constant stream of events instead of a tidy nightly dump. Your pipeline now has to ingest, transform, and serve data while transactions are still happening. Add ISO 20022 into the mix and the pressure climbs. ISO 20022 messages are rich. They carry far more structured detail than the old formats, which is wonderful for analytics and miserable for a pipeline that was never designed to parse that much context at speed. This is not a fringe concern either. Swift reported that by the time its MT/ISO 20022 coexistence period closed in November 2025, around 80% of daily traffic was already running on the ISO 20022 format, with more than 3.1 million of these messages exchanged every day. The rich-data era is the default now, not the roadmap. Then there is the fraud-scoring window. Fraud models need fresh features. Account behaviour over the last few minutes, velocity checks, device signals. If your pipeline takes thirty seconds to surface that data, the fraud decision is already too late. You are essentially detecting fraud after the loss. That gap between when data is created and when it becomes usable is the silent killer in most payment systems. And the cost of getting it wrong runs