Your Service Map Is Lying

You attach the OpenTelemetry Java agent, point it at a collector, and within minutes Grafana is drawing a service map you never drew. A box for each service, arrows between them, latency on every edge. It feels like magic, and — more dangerously — it feels complete . "The agent traces everything" is the sentence repeated in every onboarding doc. This is the story of the moment that sentence stopped being true on my platform, why I'm glad it did, and the difference between a system that is working and a system you can actually see . The flow everyone trusts The platform is an event-driven set of Spring Boot services: an API gateway in front, a user-service backed by MySQL, a notification-service backed by PostgreSQL, and Kafka carrying events between them. A user is created, an event is published, a notification is sent. I didn't want to draw that topology. A hand-drawn architecture diagram is documentation that drifts — true the day you commit it, slightly wrong a month later, actively misleading after a quarter. I wanted the dependency graph generated from live traffic , so it would always reflect what the system actually does. Grafana Tempo does exactly this. Its service-graphs processor reads matched client/server span pairs out of trace data and emits a metric — traces_service_graph_request_total — that Grafana renders as a node graph. No edge is ever wired by hand. The topology is derived, continuously, from real spans. The edge that wasn't there I generated the graph and the synchronous edges lit up immediately: api-gateway → user-service user-service → MySQL notification-service → PostgreSQL Then I looked for the one edge I actually cared about — user-service → notification-service , the asynchronous hop over Kafka. It wasn't there. The naive conclusion (and why it's wrong) The tempting read is immediate and obvious: the async hop is broken. The event isn't getting across. Go debug the consumer. So I checked. And the consumer was completely fine. notification