View Source KafkaEx.Producer.Partitioner behaviour (kafka_ex v1.0.0-rc.1)
Behaviour definition for partitioners that assign partitions for produce requests.
A partitioner is responsible for determining which partition a message should be sent to when no explicit partition is provided.
Implementing a Custom Partitioner
To implement a custom partitioner, create a module that implements the
assign_partition/4 callback:
defmodule MyApp.RoundRobinPartitioner do
@behaviour KafkaEx.Producer.Partitioner
def assign_partition(_topic, _key, _value, partition_count) do
# Simple round-robin - in practice you'd want state for this
:rand.uniform(partition_count) - 1
end
endThen configure it in your application:
config :kafka_ex, partitioner: MyApp.RoundRobinPartitionerBuilt-in Partitioners
KafkaEx.Producer.Partitioner.Default- Uses murmur2 hash of key for consistent partitioning, or random selection when no key is provided.
Summary
Callbacks
Assigns a partition for the given message.
Functions
Returns the configured partitioner module.
Callbacks
@callback assign_partition( topic :: String.t(), key :: binary() | nil, value :: binary(), partition_count :: pos_integer() ) :: non_neg_integer()
Assigns a partition for the given message.
Parameters
topic- The topic namekey- The message key (may be nil)value- The message valuepartition_count- The number of partitions for the topic
Returns
The partition number (0-indexed) to send the message to.
Functions
@spec get_partitioner() :: module()
Returns the configured partitioner module.
Defaults to KafkaEx.Producer.Partitioner.Default if not configured.