View Source KafkaEx.Consumer.ConsumerGroup.Manager (kafka_ex v1.0.0-rc.1)

Manages membership in a Kafka consumer group.

This module implements the Kafka consumer group protocol, handling:

  • Group Membership: Joining and leaving consumer groups via JoinGroup/LeaveGroup requests
  • Partition Assignment: Coordinating partition distribution among group members
  • Rebalancing: Triggering rebalances when members join/leave or partitions change
  • Heartbeats: Maintaining group membership through periodic heartbeat messages

Consumer Group Protocol Flow

  1. Join Phase: Send JoinGroupRequest to group coordinator. The coordinator blocks until all members have joined, then elects a leader.

  2. Sync Phase: The leader computes partition assignments and sends them via SyncGroupRequest. All members receive their assignments in the response.

  3. Consume Phase: Members consume from assigned partitions while sending periodic heartbeats to maintain membership.

  4. Rebalance: When the coordinator signals a rebalance (via heartbeat response), members stop consuming, commit offsets, and rejoin the group.

Options

The following options can be passed when starting a consumer group:

  • :heartbeat_interval - Interval between heartbeats in ms (default: 5000)
  • :session_timeout - Session timeout in ms (default: 30000)
  • :session_timeout_padding - Extra time added to request timeouts (default: 10000)
  • :rebalance_timeout - Time allowed for consumers to rejoin during rebalance (default: session_timeout * 3)
  • :partition_assignment_callback - Function for custom partition assignment (default: round-robin)

Options can also be configured globally via application config under :kafka_ex.

Summary

Functions

Returns a specification to start this module under a supervisor.

Callback implementation for GenServer.init/1.

Types

@type assignments() :: [{binary(), integer()}]

Functions

Returns a specification to start this module under a supervisor.

See Supervisor.

Callback implementation for GenServer.init/1.