Very soon, Apache Kafka® will no longer need ZooKeeper! With KIP-500, Kafka will include its built-in consensus layer, removing the ZooKeeper dependency altogether. The next significant milestone in this effort is coming in Apache Kafka 2.8.0. You will have early access to the new code and spin up a development version of Kafka without ZooKeeper’s opportunity to play with the Raft implementation as the distributed consensus algorithm.
Ultimately, removing ZooKeeper simplifies overall infrastructure design and operational workflows for your Kafka deployments. In this talk, Viktor will provide a brief overview of what limitations Zookeeper imposed on Kafka architecture, what will be changed, and the current status of Zookeeper removal.