You can go your way: Kafka without Zookeeper

Ka ka without Zookeeper You can go your way | @con luentinc f f @gamussa

@gamussa | #JPoint | @confluentinc

★gamov.dev/twitter ★gamov.dev/telegram ★gamov.dev/youtube @gamussa | #JPoint | @confluentinc

Agenda What we are going to talk about @gamussa | #JPoint | @confluentinc

Agenda What we are going to talk about • Brief history of Ka ka f @gamussa | #JPoint | @confluentinc

Agenda What we are going to talk about • Brief history of Ka @gamussa | #JPoint | @confluentinc

f ka Past and Future

replication @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

streams @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

exactly once @gamussa | #JPoint | @confluentinc

ksqlDB @gamussa | #JPoint | @confluentinc

KIP 500 @gamussa | #JPoint | @confluentinc

KIP-455: Create an Administrative API for Replica Reassignment KIP-497: Add inter-broker API to alter ISR KIP-543: Expand Con igCommand’s non-ZK functionality KIP-555: Deprecate Direct Zookeeper access in Ka ka Administrative Tools KIP-589: Add API to update Replica state in Controller KIP-590: Redirect Zookeeper Mutation Protocols to The Controller KIP-595: A Raft Protocol for the Metadata Quorum KIP-631: The Quorum-based Ka ka Controller f f f @gamussa | #JPoint | @confluentinc

How Ka ka uses ZooKeeper f @gamussa | #JPoint | @confluentinc

Clients @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

producer consumer streams connect @gamussa | #JPoint | @confluentinc

admin client @gamussa | #JPoint | @confluentinc

admin client /bin tools @gamussa | #JPoint | @confluentinc

quota management replica reassignment admin client /bin tools @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

Brokers @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

ISR changes event notifications broker registration @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

Client Compatibility @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

CreateTopic @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

AlterConfig @gamussa | #JPoint | @confluentinc

35 @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

AlterConfig @gamussa | #JPoint | @confluentinc

KIP-500: Current State @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

Metadata Quorum @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

Leader (Controller) @gamussa | #JPoint | @confluentinc

Follower Leader (Controller) Follower @gamussa | #JPoint | @confluentinc

Follower Leader (Controller) Follower Observers @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

Data Nodes Metadata Nodes @gamussa | #JPoint | @confluentinc

Dedicated Deployment Data Nodes Metadata Nodes @gamussa | #JPoint | @confluentinc

Data Nodes Metadata Nodes @gamussa | #JPoint | @confluentinc

Data Nodes @gamussa | #JPoint | @confluentinc

Data/Metadata Nodes @gamussa | #JPoint | @confluentinc

Mixed Deployment Data/Metadata Nodes @gamussa | #JPoint | @confluentinc

Mixed Deployment @gamussa | #JPoint | @confluentinc

Mixed Deployment Observer Promotion @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

Single Node Deployment @gamussa | #JPoint | @confluentinc

Single Node Deployment MM MM MM @gamussa | #JPoint | @confluentinc

KRaft: Ka kaesque Raft f @gamussa | #JPoint | @confluentinc

https://www.kafka-summit.org/sessions/a-kafkaesque-raft-protocol @gamussa | #JPoint | @confluentinc

Writes Fencing Ka ka Raft Single Leader Single Leader Monotonically increasing epoch Monotonically increasing term Log reconciliation Offset and epoch Term and index Push/Pull Pull Push Commit Semantics ISR Majority Leader Election From ISR through Zookeeper Majority Fault Tolerance F+1 2F+1 f @gamussa | #JPoint | @confluentinc

@gamussa | #JPoint | @confluentinc

STOP! Demo time! @gamussa | #JPoint | @confluentinc

Want to learn more about Ka ka? f @gamussa | #JPoint | @confluentinc

Learn Ka ka. Watch full version https://gamov.dev/developer Start building with Apache Ka ka at Con luent Developer. f f f f f Con luent Developer developer.con luent.io

As Always Have A Nice Day @gamussa | @confluentinc