Kaffe.WorkerManager (Kaffe v1.25.0) View Source

Manage topic/partition-to-worker assignments. Subscribers get workers from here.

This process manages the workers, while delegating to Kaffe.WorkerSupervisor to start each worker under supervision.

Workers are allocated based on a configured strategy (worker_allocation_strategy in the consumer config): either one worker per topic/partition, or one worker for a partition across topics.

The first strategy is useful for higher throughput and increased flexibility. The second is useful when mutliple input topics may have the same messages (identified by key) and those messages must be processsed sequentially.

The table of workers is stored in an ETS table, :kaffe_workers.

Link to this section Summary

Functions

Returns a specification to start this module under a supervisor.

Callback implementation for GenServer.init/1.

Link to this section Functions

Returns a specification to start this module under a supervisor.

See Supervisor.

Callback implementation for GenServer.init/1.

Link to this function

start_link(subscriber_name)

View Source
Link to this function

worker_for(pid, topic, partition)

View Source
Link to this function

worker_name(topic, partition)

View Source