Bringing Observability to Your Stream Processing

Abstract

Using microservices—breaking down the domain problem into multiple reusable, highly scalable, and efficient services—is rapidly becoming the de facto standard way to develop applications. However, when it comes to monitoring these services, developers often have to leverage distributed tracing technologies such as OpenTracing to make the transaction flow observable. Although this is relatively easy to accomplish while dealing with request/response microservices, it might not be that easy to bring distributed tracing to event-driven/asynchronous architectures built on Apache Kafka. This session explains how distributed tracing can be implemented in pipelines built with Apache Kafka, via usage of OpenTracing and the Kafka Interceptors API.