Testing Kafka Containers with Testcontainers: There and Back Again
A presentation at Current 2022: The Next Generation of Kafka Summit in in Austin, TX, USA by Viktor Gamov
Testing Kafka with Testcontainers There and Back Again Current, Austing TX, 2022 @gamussa | #current22 | @thekonginc
How many times you’ve heard @gamussa | #current22 | @thekonginc
How many times you’ve heard «IT WORKS ON MY MACHINE 😝» @gamussa | #current22 | @thekonginc
How many times you’ve heard @gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
docker pull confluentinc/cp-kafka:7.2.2 docker run -p 9092:9092 cp-kafka:7.2.2 @gamussa | #current22 | @thekonginc
—version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:7.2.2 ports: - “32181:32181” environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:7.2.2 ports: - “29092:29092” depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT:// localhost:29092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 @gamussa | #current22 | @thekonginc
—version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:7.2.2 ports: - “32181:32181” environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:7.2.2 ports: - “29092:29092” depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT:// localhost:29092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 @gamussa | #current22 | @thekonginc
public KafkaContainer createKafka() { return new KafkaContainer(imageName(“cp-kafka”)); } @gamussa | #current22 | @thekonginc
public KafkaContainer createKafka() { return new KafkaContainer(imageName(“cp-kafka”)); } @gamussa | #current22 | @thekonginc
VIKTOR GAMOV @gamussa | #current22 | @thekonginc
VIKTOR GAMOV • Principal Developer Advocate @ Kong @gamussa | #current22 | @thekonginc
VIKTOR GAMOV • Principal Developer Advocate @ Kong • Confluent Community Catalyst @gamussa | #current22 | @thekonginc
VIKTOR GAMOV • Principal Developer Advocate @ Kong • Confluent Community Catalyst • Java Champion @gamussa | #current22 | @thekonginc
VIKTOR GAMOV • • • • Principal Developer Advocate @ Kong Confluent Community Catalyst Java Champion Author @ Manning @gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
KAFKA IN ACTION IS OUT! Use code KAFKAA35 @gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
docker pull confluentinc/cp-kafka:7.2.2 docker run -p 9092:9092 cp-kafka:7.2.2 @gamussa | #current22 | @thekonginc
The Cure The Pain @gamussa | #current22 | @thekonginc
The Pain The Cure @gamussa | #current22 | @thekonginc
The Pain The Cure Port conflicts @gamussa | #current22 | @thekonginc
The Pain The Cure Port conflicts Port randomization @gamussa | #current22 | @thekonginc
The Pain The Cure Port conflicts Port randomization Hard-coded scenarios @gamussa | #current22 | @thekonginc
The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests @gamussa | #current22 | @thekonginc
The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests Slow test @gamussa | #current22 | @thekonginc
The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests Slow test Parallelization @gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
We think it’s a useful default option for creating a reliable environment for running tests. … Our teams have consistently found this library of programmable, lightweight and disposable containers to make functional tests more reliable. https://www.thoughtworks.com/en-us/radar/languages-and-frameworks/testcontainers @gamussa | #current22 | @thekonginc
LET’S SEE SOME CODE! @gamussa | #current22 | @thekonginc
What to do if you have no Docker? @gamussa | #current22 | @thekonginc
https://www.atomicjar.com/2021/10/docker-on-windows-and-macos @gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
LET’S SEE SOME CODE! @gamussa | #current22 | @thekonginc
How it’s started gAmUssA/testcontainers-java-module-con uent-platform fl @gamussa | #current22 | @thekonginc
How it’s going testcontainers-all-things-kafka/cp-testcontainers @gamussa | #current22 | @thekonginc
How it’s going testcontainers-all-things-kafka/cp-testcontainers-examples @gamussa | #current22 | @thekonginc
ONE MORE THING… @gamussa | #current22 | @thekonginc
https://www.thoughtworks.com/en-us/radar/languages-and-frameworks/testcontainers https://www.thoughtworks.com/radar/platforms/ka ka-api-without-ka ka f f @gamussa | #current22 | @thekonginc
@gamussa | #current22 | @thekonginc
DEVELOPMENT AND DEPLOYMENT WITH BORING SCRIPTS @gamussa | #current22 | @thekonginc
DEVELOPMENT AND EXPLORE DEPLOYMENT THE WITH WORLD OF CONTAINERS BORING SCRIPTS @gamussa | #current22 | @thekonginc
ONE MORE THING… @gamussa | #current22 | @thekonginc
LOCAL DEVELOPMENT Dev Services https://quarkus.io/guides/dev-services Test Resources https://micronaut-projects.github.io/micronaut-test-resources/latest/guide/ Autoservices https://github.com/spring-projects/spring-boot/issues/29461 @gamussa | #current22 | @thekonginc
AS ALWAYS, HAVE A NICE DAY… Viktor Gamov @ gAmUssA @gamussa | #current22 | @thekonginc
Did you ever wonder how your applications will behave once deployed to production? Sure, you have unit tests, and your test coverage is sky-high. However, you might depend on external resources like Apache Kafka or Kafka Connect connectors, kSQL, etc.
Moreover, without proper integration testing, you cannot be confident about the stability of your production environment.
In this session, Viktor talks about Testcontainers, a library (that was initially created for JVM, now exists in many languages) that provides lightweight, disposable instances of shared databases, clusters, and anything else that can run in a Docker container! After a rapid-fire introduction to the core concepts of the containers and how they can help improve integration testing, we’re going to zoom in on supported out-of-the-box containers.
You will learn how to test complex stacks like Apache Kafka-based streaming platform (or even Confluent Cloud) and other components.
Resources
The following resources were mentioned during the presentation or are useful additional information.
Buzz and feedback
Here’s what was said about this presentation on social media.
