Follower Fetching
Learn about follower fetching and how to configure a Redpanda consumer to fetch records from the closest replica.
About follower fetching
Follower fetching enables a consumer to fetch records from the closest replica of a topic partition, regardless of whether it’s a leader or a follower.
For a Redpanda cluster deployed across different data centers and availability zones (AZs), restricting a consumer to fetch only from the leader of a partition can incur greater costs and have higher latency than fetching from a follower that is geographically closer to the consumer.
With follower fetching (proposed in KIP-392), the fetch protocol is extended to support a consumer fetching from any replica. This includes Remote Read Replicas.
The first fetch from a consumer is processed by a Redpanda leader broker. The leader checks for a replica (itself or a follower) that has a rack ID that matches the consumer’s rack ID. If a replica with a matching rack ID is found, the fetch request returns records from that replica. Otherwise, the fetch is handled by the leader.
Configure follower fetching
Redpanda decides which replica a consumer fetches from. If the consumer configures its client.rack property, Redpanda by default selects a replica from the same rack as the consumer, if available.
To enable follower fetching in Redpanda, configure properties for the consumer and the Redpanda cluster and broker:
-
For a Redpanda cluster, set the
enable_rack_awarenessproperty totrue. -
For each Redpanda broker, set the
rackproperty to a rack ID. -
For each consumer, set the
client.rackproperty to a rack ID.
| Follower fetching spreads consumers across replicas to distribute the read load. If many consumers also fetch the same data concurrently, fetch read coalescing complements it by removing the redundant work within each shard. |
