Why I Publish to Kafka Only After the Transaction Commits
The bug that doesn't show up in tests — and what to do about it There is a class of bug in event-driven systems that is almost invisible in development and devastating in production: publishing a message to Kafka for data that never actually reached the database. It doesn't crash. It doesn't throw. The Kafka message goes out, the consumer picks it up, and it tries to process a batch that doesn't exist. Depending on your retry and error handling strategy, this can cascade silently for a long time before anyone notices. The fix is simple. The reason most people don't apply it is that the problem isn't obvious until you've seen it. The Problem: Publishing Inside the Transaction The intuitive approach is to publish to Kafka as part of the same transactional method: @Transactional public void process ( SettlementWindow window , LocalDate today , Participant participant ) { // ... FileBatch savedBatch = batchPort . save ( batch ); orderPort . updateStatusBatch ( orders ); // Publishes BEFORE the transaction commits publisherPort . publish ( savedBatch ); } This looks safe. The transaction is still open, the data is there, everything is consistent — until the transaction rolls back. If anything fails after publish() — another database update, a constraint violation, an unexpected exception — Spring rolls back the transaction. The database returns to its previous state. But Kafka already received the message. There is no rollback for Kafka. The consumer now holds a reference to a FileBatch that does not exist in the database. This is a phantom message . The Fix: afterCommit() Spring's TransactionSynchronizationManager provides a hook that fires after the transaction has successfully committed: @Transactional ( propagation = Propagation . REQUIRES_NEW ) public void process ( SettlementWindow window , LocalDate today , Participant participant ) { // ... FileBatch savedBatch = batchPort . save ( batch ); orderPort . updateStatusBatch ( orders ); // Kafka fires only after the d