Recently I studied Kafka and wanted to share my understanding.

Kafka is used for handling messages/events between different services. Here's how I understand it: A Producer sends an event/message to Kafka. The message contains things like Topic, Key-Value data, and Timestamp. Kafka stores these messages in Brokers (Kafka servers). Topics can be divided into multiple Partitions. Each partition has one Leader and multiple Followers (Replicas). All read and write operations happen through the Leader, while Replicas act as backups if a broker fails. Now Kafka does not immediately delete messages after they are consumed, unlike many traditional queues. There is a term called Offsets. You can think of an offset like the index of a message inside a partition. For example: A user places an order → payment is processed → email is sent → analytics service processes the event. Suppose during that analytics service goes down, Kafka knows which offset was last processed. When the service comes back up, it can continue from that offset instead of starting from the beginning. This is also one reason why Kafka keeps messages for some time after consumption. Any corrections? Is there anything else I should know about this topic? Please let me know. submitted by /u/No-Resolution-4054 [link] [留言]