I don’t always test my streams, but when I do I do it in production
A presentation at jconf.dev by Viktor Gamov
I don’t always test my streams, but when I do I do it in production August 2020 / Kafka Summit @gamussa @gamussa || @jconf.dev @jconf.dev || @confluentinc @confluentinc
@gamussa | @jconf.dev | @confluentinc
Special thanks @inponomarev @gamussa @bsideup | @jconf.dev | @confluentinc
https://gamov.dev/jdevconf2020 @gamussa | @jconf.dev | @confluentinc
Agenda 01 04 Kafka Apps Testing Streams 02 05 Testing basics Testing ksqlDB 03 06 Can we better? Conclusion What are those? TopologyTestDriver vs TestContainers Run Kafka and CP locally Ksql-test-runner vs TestContainers Always Be Testing Test with TestContainers @gamussa | @jconf.dev | @confluentinc
Preface @gamussa | @jconf.dev | @confluentinc
Senior Developer: - Where did you get this code? Junior Developer: - From Stackoverflow Senior Developer: - from question or answer? @gamussa | @jconf.dev | @confluentinc
An example from my life: Kafka Tutorials website @gamussa | @jconf.dev | @confluentinc
Literate Programming… Do you speak it… https://en.wikipedia.org/wiki/Literate_programming @gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
Kafka Apps aka stream processing @gamussa | @jconf.dev | @confluentinc
Stream Processing is the toolset for dealing with events as they move! @gamussa | @jconf.dev | @confluentinc
Producers Kafka Consumers @gamussa | @jconf.dev | @confluentinc
What are Kafka Apps? My App @gamussa | @jconf.dev | @confluentinc
Schema, WUT? @gamussa | @jconf.dev | @confluentinc
Let’s see some code @gamussa | @jconf.dev | @confluentinc
Kafka Streams for stream processing @gamussa | @jconf.dev | @confluentinc
the KAFKA STREAMS API is a JAVA API to BUILD REAL-TIME APPLICATIONS @gamussa | @jconf.dev | @confluentinc
https://kafka.apache.org/documentation/streams/developer-guide/testing.html @gamussa | @jconf.dev | @confluentinc
Topology Test driver @gamussa | @jconf.dev | @confluentinc
• TopologyTestDriver • Unit test your Kafka Streams Topology • Doesn’t Require Kafka infra to run • Convenience methods to define input and output • Convenience methods to provide fixtures and validate the results @gamussa | @jconf.dev | @confluentinc
Let’s see some code @gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
When TTD doesn’t help? https://gamov.dev/livestreams-s01e05 @gamussa | @jconf.dev | @confluentinc
Ksqldb for Stream processing @gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
Let’s see some code @gamussa | @jconf.dev | @confluentinc
Conclusion @gamussa | @jconf.dev | @confluentinc
What did you learn today? • What’s Kafka apps apps and how to test them • Producer / Consumer API • Kafka Streams • ksqlDB • How containers and TestContainer help a lot @gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
Want to learn more? @gamussa | #springone | @confluentinc
Learn Kafka. Start building with Apache Kafka at Confluent Developer. developer.confluent.io Watch full version https://gamov.dev/developer
Confluent Community Slack A vibrant community of over 16,000 members Come along and discuss Apache Kafka and Confluent Platform on dedicated channels including #ksqlDB, #connect, #clients, and more http://cnfl.io/slack @gamussa | @jconf.dev | @confluentinc
@gamussa | @jconf.dev | @confluentinc
In this presentation, Viktor will cover all important aspects of testing stream-processing and event-driven applications
