Ka ka without Zookeeper You can go your way | @con luentinc f f @gamussa
Slide 1
Slide 2
Slide 3
Slide 4
@gamussa | #JPoint | @confluentinc
Slide 5
★gamov.dev/twitter ★gamov.dev/telegram ★gamov.dev/youtube @gamussa | #JPoint | @confluentinc
Slide 6
Agenda What we are going to talk about @gamussa | #JPoint | @confluentinc
Slide 7
Agenda What we are going to talk about • Brief history of Ka ka f @gamussa | #JPoint | @confluentinc
Slide 8
Agenda What we are going to talk about • Brief history of Ka @gamussa | #JPoint | @confluentinc
Slide 9
Slide 10
Slide 11
Slide 12
Slide 13
Slide 14
f ka Past and Future
Slide 15
replication @gamussa | #JPoint | @confluentinc
Slide 16
replication @gamussa | #JPoint | @confluentinc
Slide 17
@gamussa | #JPoint | @confluentinc
Slide 18
streams @gamussa | #JPoint | @confluentinc
Slide 19
@gamussa | #JPoint | @confluentinc
Slide 20
exactly once @gamussa | #JPoint | @confluentinc
Slide 21
exactly once @gamussa | #JPoint | @confluentinc
Slide 22
ksqlDB @gamussa | #JPoint | @confluentinc
Slide 23
KIP 500 @gamussa | #JPoint | @confluentinc
Slide 24
KIP 500 @gamussa | #JPoint | @confluentinc
Slide 25
KIP-455: Create an Administrative API for Replica Reassignment KIP-497: Add inter-broker API to alter ISR KIP-543: Expand Con igCommand’s non-ZK functionality KIP-555: Deprecate Direct Zookeeper access in Ka ka Administrative Tools KIP-589: Add API to update Replica state in Controller KIP-590: Redirect Zookeeper Mutation Protocols to The Controller KIP-595: A Raft Protocol for the Metadata Quorum KIP-631: The Quorum-based Ka ka Controller f f f @gamussa | #JPoint | @confluentinc
Slide 26
How Ka ka uses ZooKeeper f @gamussa | #JPoint | @confluentinc
Slide 27
Clients @gamussa | #JPoint | @confluentinc
Slide 28
@gamussa | #JPoint | @confluentinc
Slide 29
@gamussa | #JPoint | @confluentinc
Slide 30
producer consumer streams connect @gamussa | #JPoint | @confluentinc
Slide 31
admin client @gamussa | #JPoint | @confluentinc
Slide 32
admin client /bin tools @gamussa | #JPoint | @confluentinc
Slide 33
quota management replica reassignment admin client /bin tools @gamussa | #JPoint | @confluentinc
Slide 34
@gamussa | #JPoint | @confluentinc
Slide 35
@gamussa | #JPoint | @confluentinc
Slide 36
@gamussa | #JPoint | @confluentinc
Slide 37
Brokers @gamussa | #JPoint | @confluentinc
Slide 38
@gamussa | #JPoint | @confluentinc
Slide 39
@gamussa | #JPoint | @confluentinc
Slide 40
@gamussa | #JPoint | @confluentinc
Slide 41
@gamussa | #JPoint | @confluentinc
Slide 42
@gamussa | #JPoint | @confluentinc
Slide 43
ISR changes event notifications broker registration @gamussa | #JPoint | @confluentinc
Slide 44
@gamussa | #JPoint | @confluentinc
Slide 45
@gamussa | #JPoint | @confluentinc
Slide 46
Client Compatibility @gamussa | #JPoint | @confluentinc
Slide 47
@gamussa | #JPoint | @confluentinc
Slide 48
@gamussa | #JPoint | @confluentinc
Slide 49
@gamussa | #JPoint | @confluentinc
Slide 50
CreateTopic @gamussa | #JPoint | @confluentinc
Slide 51
CreateTopic @gamussa | #JPoint | @confluentinc
Slide 52
CreateTopic @gamussa | #JPoint | @confluentinc
Slide 53
@gamussa | #JPoint | @confluentinc
Slide 54
AlterConfig @gamussa | #JPoint | @confluentinc
Slide 55
AlterConfig @gamussa | #JPoint | @confluentinc
Slide 56
AlterConfig @gamussa | #JPoint | @confluentinc
Slide 57
35 @gamussa | #JPoint | @confluentinc
Slide 58
@gamussa | #JPoint | @confluentinc
Slide 59
AlterConfig @gamussa | #JPoint | @confluentinc
Slide 60
AlterConfig @gamussa | #JPoint | @confluentinc
Slide 61
AlterConfig @gamussa | #JPoint | @confluentinc
Slide 62
AlterConfig @gamussa | #JPoint | @confluentinc
Slide 63
KIP-500: Current State @gamussa | #JPoint | @confluentinc
Slide 64
@gamussa | #JPoint | @confluentinc
Slide 65
Metadata Quorum @gamussa | #JPoint | @confluentinc
Slide 66
@gamussa | #JPoint | @confluentinc
Slide 67
@gamussa | #JPoint | @confluentinc
Slide 68
Leader (Controller) @gamussa | #JPoint | @confluentinc
Slide 69
Follower Leader (Controller) Follower @gamussa | #JPoint | @confluentinc
Slide 70
Follower Leader (Controller) Follower Observers @gamussa | #JPoint | @confluentinc
Slide 71
@gamussa | #JPoint | @confluentinc
Slide 72
Data Nodes Metadata Nodes @gamussa | #JPoint | @confluentinc
Slide 73
Dedicated Deployment Data Nodes Metadata Nodes @gamussa | #JPoint | @confluentinc
Slide 74
Data Nodes Metadata Nodes @gamussa | #JPoint | @confluentinc
Slide 75
Data Nodes @gamussa | #JPoint | @confluentinc
Slide 76
Data/Metadata Nodes @gamussa | #JPoint | @confluentinc
Slide 77
Mixed Deployment Data/Metadata Nodes @gamussa | #JPoint | @confluentinc
Slide 78
Mixed Deployment @gamussa | #JPoint | @confluentinc
Slide 79
Mixed Deployment @gamussa | #JPoint | @confluentinc
Slide 80
Mixed Deployment Observer Promotion @gamussa | #JPoint | @confluentinc
Slide 81
Mixed Deployment Observer Promotion @gamussa | #JPoint | @confluentinc
Slide 82
@gamussa | #JPoint | @confluentinc
Slide 83
Single Node Deployment @gamussa | #JPoint | @confluentinc
Slide 84
Single Node Deployment @gamussa | #JPoint | @confluentinc
Slide 85
Single Node Deployment MM MM MM @gamussa | #JPoint | @confluentinc
Slide 86
KRaft: Ka kaesque Raft f @gamussa | #JPoint | @confluentinc
Slide 87
https://www.kafka-summit.org/sessions/a-kafkaesque-raft-protocol @gamussa | #JPoint | @confluentinc
Slide 88
Writes Fencing Ka ka Raft Single Leader Single Leader Monotonically increasing epoch Monotonically increasing term Log reconciliation Offset and epoch Term and index Push/Pull Pull Push Commit Semantics ISR Majority Leader Election From ISR through Zookeeper Majority Fault Tolerance F+1 2F+1 f @gamussa | #JPoint | @confluentinc
Slide 89
@gamussa | #JPoint | @confluentinc
Slide 90
STOP! Demo time! @gamussa | #JPoint | @confluentinc
Slide 91
Want to learn more about Ka ka? f @gamussa | #JPoint | @confluentinc
Slide 92
Learn Ka ka. Watch full version https://gamov.dev/developer Start building with Apache Ka ka at Con luent Developer. f f f f f Con luent Developer developer.con luent.io
Slide 93
As Always Have A Nice Day @gamussa | @confluentinc
Slide 94
