An opinionated, highly specific, Elixir wrapper around brod: the Erlang Kafka client. :coffee

Consume messages from Kafka and pass to a given local module

Running state for the consumer

This is the main process for launching group members and workers

Consume messages from a Kafka topic for a consumer group

The top-level supervisor for group members and subscribers

A GenServer for producing messages to a given Kafka topic

• client - the brod client to call for producing
• topics - a list of configured topics
• partition_strategy - the strategy to determine the next partition

• partition_details - a map of partition details keyed by topic

  • total - the number of partitions
  • partition - the next partition to produce to

Consume messages from a single partition of a single Kafka topic

A worker is assigned a single partition across topics for the client. This is so we effectively serialize the processing of any single key across topics

Manage partition-to-worker assignments. Subscribers get workers from here

Supervise the individual workers