Kafka on Kubernetes: Does It Really Have To Be “The Hard Way”?

Kafka on Kubernetes: Does it really have to be «The Hard Way»? May, 2019 / Texas, 2019 @gamussa @gamussa || #PerconaLive #PerconaLive || @ConfluentINc @ConfluentINc

Raffle, yeah 🚀 Follow @gamussa @confluentinc 📸🖼👬 Tag @gamussa With #PerconaLive

3 Evolution of #devkafkaops ansible/chef Docker Kubernetes Shell scripts @gamussa | #PerconaLive | @ConfluentINc

4 @gamussa | #PerconaLive | @ConfluentINc

5 🙋 @gamussa | #PerconaLive | @ConfluentINc

6 🙋 Who run stateless workloads in Kubernetes? Who thinks it’s a good idea? Who run stateful workloads in Kubernetes? Who thinks it’s a good idea? @gamussa | #PerconaLive | @ConfluentINc

7 kafkaesque world of Kafka on Kubernetes @gamussa | #PerconaLive | @ConfluentINc

8 #devkafkaops Well, it’s tricky © Translating an existing architecture to Kubernetes External access to brokers and other components Persistent Storage options on prem and clouds Security Configuration and Upgrades @gamussa | #PerconaLive | @ConfluentINc

9 But I just want to deploy kafka @gamussa | #PerconaLive | @ConfluentINc

10 Workloads Deployment @gamussa | #PerconaLive | @ConfluentINc

11 https://cnfl.io/helm_video @gamussa | #PerconaLive | @ConfluentINc

12 We will use StatefulSets with OrderedReady @gamussa | #PerconaLive | @ConfluentINc

13 We need SRE / Operator knowledge to manage the platform. You need Operator! @gamussa | #PerconaLive | @ConfluentINc

14 @gamussa | #PerconaLive | @ConfluentINc

15 Show me your Operator @gamussa | #PerconaLive | @ConfluentINc

16 KLyfft - Kafka app on Kubernetes 2 Web Apps powered by a microservice each Apps are a ride sharing driver and rider location tracking Rider hails a ride and the driver drives towards the rider Real time location events powered by Kafka on Kubernetes Rider @gamussa | #PerconaLive | @ConfluentINc Driver

17 Demo @gamussa | #PerconaLive | @ConfluentINc

18 DO KAFKA ON KUBERNETES DEMO AND EVERYONE LOOSES THEIR MIND @gamussa | #PerconaLive | @ConfluentINc

19 Confluent Operator - Automated Security Configuration SASL PLAIN and Mutual TLS Authentication Automate configuration of truststores and keystores with secret objects Automate configuration of Kafka and all Confluent Platform Components @gamussa | #PerconaLive | @ConfluentINc

20 Be like Justin! @gamussa | #PerconaLive | @ConfluentINc

Rolling Upgrade Kafka Broker Upgrades: 1. Stop the broker, upgrade Kafka 2. Wait for Partition Leader reassignment 3. Start the upgraded broker 4. Wait for zero underreplicated partitions 5. Upgrade the next broker @gamussa | #PerconaLive | @ConfluentINc 21

22 Want to see advanced deployment? vs. @gamussa | #PerconaLive | @ConfluentINc

23 http://gamov.dev/ksummit_nyc19_k8s @gamussa | #PerconaLive | @ConfluentINc

24 ● We are in private Preview GA Plans Release now ● 24 customers testing the Operator in Preview: ● Global customers ● Banks, Fin Tech, Retailers, Consumer Tech ● We are in the final stages of Preview and about to launch soon @gamussa | #PerconaLive | @ConfluentINc

Thanks! @gamussa viktor@confluent.io https://slackpass.io/confluentcommunity #kubernetes @gamussa | @ #PerconaLive | @ConfluentINc

26