Testing Kafka Containers with Testcontainers: There and Back Again

A presentation at Kafka Summit London 2022 in April 2022 in London, UK by Viktor Gamov

Slide 1

Slide 1

Testing Kafka with Testcontainers There and Back Again Kafka Summit, London, 2022 @gamussa | #kafkasummit | @thekonginc

Slide 2

Slide 2

How many times you’ve heard @gamussa | #kafkasummit | @thekonginc

Slide 3

Slide 3

How many times you’ve heard «IT WORKS ON MY MACHINE 😝» @gamussa | #kafkasummit | @thekonginc

Slide 4

Slide 4

How many times you’ve heard @gamussa | #kafkasummit | @thekonginc

Slide 5

Slide 5

@gamussa | #kafkasummit | @thekonginc

Slide 6

Slide 6

@gamussa | #kafkasummit | @thekonginc

Slide 7

Slide 7

@gamussa | #kafkasummit | @thekonginc

Slide 8

Slide 8

docker pull confluentinc/cp-kafka:7.0.1 docker run -p 9092:9092 cp-kafka:7.0.1 @gamussa | #kafkasummit | @thekonginc

Slide 9

Slide 9

—version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:7.0.1 ports: - “32181:32181” environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:7.0.1 ports: - “29092:29092” depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT:// localhost:29092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 @gamussa | #kafkasummit | @thekonginc

Slide 10

Slide 10

—version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:7.0.1 ports: - “32181:32181” environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:7.0.1 ports: - “29092:29092” depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT:// localhost:29092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 @gamussa | #kafkasummit | @thekonginc

Slide 11

Slide 11

public KafkaContainer createKafka() { return new KafkaContainer(imageName(“cp-kafka”)); } @gamussa | #kafkasummit | @thekonginc

Slide 12

Slide 12

public KafkaContainer createKafka() { return new KafkaContainer(imageName(“cp-kafka”)); } @gamussa | #kafkasummit | @thekonginc

Slide 13

Slide 13

VIKTOR GAMOV Principal Developer Advocate @ Kong @gamussa | #kafkasummit | @thekonginc

Slide 14

Slide 14

@gamussa | #kafkasummit | @thekonginc

Slide 15

Slide 15

KAFKA IN ACTION IS OUT! Use code KAFKAA35 @gamussa | #kafkasummit | @thekonginc

Slide 16

Slide 16

@gamussa | #kafkasummit | @thekonginc

Slide 17

Slide 17

@gamussa | #kafkasummit | @thekonginc

Slide 18

Slide 18

docker pull confluentinc/cp-kafka:7.0.1 docker run -p 9092:9092 cp-kafka:7.0.1 @gamussa | #kafkasummit | @thekonginc

Slide 19

Slide 19

The Cure The Pain @gamussa | #kafkasummit | @thekonginc

Slide 20

Slide 20

The Pain The Cure @gamussa | #kafkasummit | @thekonginc

Slide 21

Slide 21

The Pain The Cure Port conflicts @gamussa | #kafkasummit | @thekonginc

Slide 22

Slide 22

The Pain The Cure Port conflicts Port randomization @gamussa | #kafkasummit | @thekonginc

Slide 23

Slide 23

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios @gamussa | #kafkasummit | @thekonginc

Slide 24

Slide 24

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests @gamussa | #kafkasummit | @thekonginc

Slide 25

Slide 25

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests Slow test @gamussa | #kafkasummit | @thekonginc

Slide 26

Slide 26

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests Slow test Parallelization @gamussa | #kafkasummit | @thekonginc

Slide 27

Slide 27

@gamussa | #kafkasummit | @thekonginc

Slide 28

Slide 28

We think it’s a useful default option for creating a reliable environment for running tests. … Our teams have consistently found this library of programmable, lightweight and disposable containers to make functional tests more reliable. https://www.thoughtworks.com/en-us/radar/languages-and-frameworks/testcontainers @gamussa | #kafkasummit | @thekonginc

Slide 29

Slide 29

LET’S SEE SOME CODE! @gamussa | #kafkasummit | @thekonginc

Slide 30

Slide 30

What to do if you have no Docker? @gamussa | #kafkasummit | @thekonginc

Slide 31

Slide 31

https://www.atomicjar.com/2021/10/docker-on-windows-and-macos @gamussa | #kafkasummit | @thekonginc

Slide 32

Slide 32

@gamussa | #kafkasummit | @thekonginc

Slide 33

Slide 33

LET’S SEE SOME CODE! @gamussa | #kafkasummit | @thekonginc

Slide 34

Slide 34

How it’s started gAmUssA/testcontainers-java-module-con uent-platform fl @gamussa | #kafkasummit | @thekonginc

Slide 35

Slide 35

How it’s going testcontainers-all-things-kafka/cp-testcontainers @gamussa | #kafkasummit | @thekonginc

Slide 36

Slide 36

How it’s going testcontainers-all-things-kafka/cp-testcontainers-examples @gamussa | #kafkasummit | @thekonginc

Slide 37

Slide 37

ONE MORE THING… @gamussa | #kafkasummit | @thekonginc

Slide 38

Slide 38

https://www.thoughtworks.com/en-us/radar/languages-and-frameworks/testcontainers https://www.thoughtworks.com/radar/platforms/ka ka-api-without-ka ka f f @gamussa | #kafkasummit | @thekonginc

Slide 39

Slide 39

@gamussa | #kafkasummit | @thekonginc

Slide 40

Slide 40

DEVELOPMENT AND DEPLOYMENT WITH BORING SCRIPTS @gamussa | #kafkasummit | @thekonginc

Slide 41

Slide 41

DEVELOPMENT AND EXPLORE DEPLOYMENT THE WITH WORLD OF CONTAINERS BORING SCRIPTS @gamussa | #kafkasummit | @thekonginc

Slide 42

Slide 42

Slide 43

Slide 43

AS ALWAYS, HAVE A NICE DAY… Viktor Gamov @ gAmUssA @gamussa | #kafkasummit | @thekonginc