Part 2 · 1 chapters · ~8 min

Replication, ISR and KRaft

Leaders and followers, the in-sync replica set, the high watermark and commit, acks and min.insync.replicas, leader election and unclean elections, rack awareness, KRaft controllers replacing ZooKeeper, and what can be lost under each configuration.

4

Durability settings

producer ackswaits forcan lose
acks=0nothinganything (fire and forget)
acks=1the leader's writerecords if the leader fails before followers copy them
acks=all + min.insync.replicas=2all in-sync replicas, at least 2nothing short of losing every in-sync replica

Kafka acknowledges after the write reaches the page cache of the replicas, not after fsync: durability comes from replication across machines (and racks or zones via broker.rack), not from each disk.

REPLICATION, ISR AND KRAFT
a leader, followers that keep up, and a Raft quorum managing it all
leaderbroker 1, partition 0followerbroker 2 (in sync)followerbroker 3 (lagging, out of ISR)KRaft controllers × 3Raft metadata quorumhigh watermarkcommitted offsetproduceracks=all
swipe the figure sideways, or tap expand for full screen
1/5
leaders and followers
Each partition has a leader that takes reads and writes; followers fetch from the leader to stay in sync. Replication factor 3 is the production standard.
leader + followers per partitionreplication factor 3