Testing Kafka Containers with Testcontainers: There and Back Again

A presentation at Current 2022: The Next Generation of Kafka Summit in October 2022 in Austin, TX, USA by Viktor Gamov

Slide 1

Slide 1

Testing Kafka with Testcontainers There and Back Again Current, Austing TX, 2022 @gamussa | #current22 | @thekonginc

Slide 2

Slide 2

How many times you’ve heard @gamussa | #current22 | @thekonginc

Slide 3

Slide 3

How many times you’ve heard «IT WORKS ON MY MACHINE 😝» @gamussa | #current22 | @thekonginc

Slide 4

Slide 4

How many times you’ve heard @gamussa | #current22 | @thekonginc

Slide 5

Slide 5

@gamussa | #current22 | @thekonginc

Slide 6

Slide 6

@gamussa | #current22 | @thekonginc

Slide 7

Slide 7

@gamussa | #current22 | @thekonginc

Slide 8

Slide 8

docker pull confluentinc/cp-kafka:7.2.2 docker run -p 9092:9092 cp-kafka:7.2.2 @gamussa | #current22 | @thekonginc

Slide 9

Slide 9

—version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:7.2.2 ports: - “32181:32181” environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:7.2.2 ports: - “29092:29092” depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT:// localhost:29092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 @gamussa | #current22 | @thekonginc

Slide 10

Slide 10

—version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:7.2.2 ports: - “32181:32181” environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:7.2.2 ports: - “29092:29092” depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT:// localhost:29092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 @gamussa | #current22 | @thekonginc

Slide 11

Slide 11

public KafkaContainer createKafka() { return new KafkaContainer(imageName(“cp-kafka”)); } @gamussa | #current22 | @thekonginc

Slide 12

Slide 12

public KafkaContainer createKafka() { return new KafkaContainer(imageName(“cp-kafka”)); } @gamussa | #current22 | @thekonginc

Slide 13

Slide 13

VIKTOR GAMOV @gamussa | #current22 | @thekonginc

Slide 14

Slide 14

VIKTOR GAMOV • Principal Developer Advocate @ Kong @gamussa | #current22 | @thekonginc

Slide 15

Slide 15

VIKTOR GAMOV • Principal Developer Advocate @ Kong • Confluent Community Catalyst @gamussa | #current22 | @thekonginc

Slide 16

Slide 16

VIKTOR GAMOV • Principal Developer Advocate @ Kong • Confluent Community Catalyst • Java Champion @gamussa | #current22 | @thekonginc

Slide 17

Slide 17

VIKTOR GAMOV • • • • Principal Developer Advocate @ Kong Confluent Community Catalyst Java Champion Author @ Manning @gamussa | #current22 | @thekonginc

Slide 18

Slide 18

@gamussa | #current22 | @thekonginc

Slide 19

Slide 19

@gamussa | #current22 | @thekonginc

Slide 20

Slide 20

@gamussa | #current22 | @thekonginc

Slide 21

Slide 21

KAFKA IN ACTION IS OUT! Use code KAFKAA35 @gamussa | #current22 | @thekonginc

Slide 22

Slide 22

@gamussa | #current22 | @thekonginc

Slide 23

Slide 23

@gamussa | #current22 | @thekonginc

Slide 24

Slide 24

docker pull confluentinc/cp-kafka:7.2.2 docker run -p 9092:9092 cp-kafka:7.2.2 @gamussa | #current22 | @thekonginc

Slide 25

Slide 25

The Cure The Pain @gamussa | #current22 | @thekonginc

Slide 26

Slide 26

The Pain The Cure @gamussa | #current22 | @thekonginc

Slide 27

Slide 27

The Pain The Cure Port conflicts @gamussa | #current22 | @thekonginc

Slide 28

Slide 28

The Pain The Cure Port conflicts Port randomization @gamussa | #current22 | @thekonginc

Slide 29

Slide 29

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios @gamussa | #current22 | @thekonginc

Slide 30

Slide 30

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests @gamussa | #current22 | @thekonginc

Slide 31

Slide 31

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests Slow test @gamussa | #current22 | @thekonginc

Slide 32

Slide 32

The Pain The Cure Port conflicts Port randomization Hard-coded scenarios Data-driven tests Slow test Parallelization @gamussa | #current22 | @thekonginc

Slide 33

Slide 33

@gamussa | #current22 | @thekonginc

Slide 34

Slide 34

We think it’s a useful default option for creating a reliable environment for running tests. … Our teams have consistently found this library of programmable, lightweight and disposable containers to make functional tests more reliable. https://www.thoughtworks.com/en-us/radar/languages-and-frameworks/testcontainers @gamussa | #current22 | @thekonginc

Slide 35

Slide 35

Slide 36

Slide 36

Slide 37

Slide 37

Slide 38

Slide 38

Slide 39

Slide 39

Slide 40

Slide 40

LET’S SEE SOME CODE! @gamussa | #current22 | @thekonginc

Slide 41

Slide 41

What to do if you have no Docker? @gamussa | #current22 | @thekonginc

Slide 42

Slide 42

https://www.atomicjar.com/2021/10/docker-on-windows-and-macos @gamussa | #current22 | @thekonginc

Slide 43

Slide 43

@gamussa | #current22 | @thekonginc

Slide 44

Slide 44

LET’S SEE SOME CODE! @gamussa | #current22 | @thekonginc

Slide 45

Slide 45

How it’s started gAmUssA/testcontainers-java-module-con uent-platform fl @gamussa | #current22 | @thekonginc

Slide 46

Slide 46

How it’s going testcontainers-all-things-kafka/cp-testcontainers @gamussa | #current22 | @thekonginc

Slide 47

Slide 47

How it’s going testcontainers-all-things-kafka/cp-testcontainers-examples @gamussa | #current22 | @thekonginc

Slide 48

Slide 48

ONE MORE THING… @gamussa | #current22 | @thekonginc

Slide 49

Slide 49

https://www.thoughtworks.com/en-us/radar/languages-and-frameworks/testcontainers https://www.thoughtworks.com/radar/platforms/ka ka-api-without-ka ka f f @gamussa | #current22 | @thekonginc

Slide 50

Slide 50

@gamussa | #current22 | @thekonginc

Slide 51

Slide 51

DEVELOPMENT AND DEPLOYMENT WITH BORING SCRIPTS @gamussa | #current22 | @thekonginc

Slide 52

Slide 52

DEVELOPMENT AND EXPLORE DEPLOYMENT THE WITH WORLD OF CONTAINERS BORING SCRIPTS @gamussa | #current22 | @thekonginc

Slide 53

Slide 53

Slide 54

Slide 54

ONE MORE THING… @gamussa | #current22 | @thekonginc

Slide 55

Slide 55

LOCAL DEVELOPMENT Dev Services https://quarkus.io/guides/dev-services Test Resources https://micronaut-projects.github.io/micronaut-test-resources/latest/guide/ Autoservices https://github.com/spring-projects/spring-boot/issues/29461 @gamussa | #current22 | @thekonginc

Slide 56

Slide 56

AS ALWAYS, HAVE A NICE DAY… Viktor Gamov @ gAmUssA @gamussa | #current22 | @thekonginc