Configuration module for KafkaEx.

Configuration Options

Add to your config/config.exs:

config :kafka_ex,
  # Required: List of broker addresses
  brokers: [{"localhost", 9092}],

  # Connection settings
  use_ssl: false,
  ssl_options: [],
  # sasl: %{mechanism: :plain, username: "user", password: "pass"},

  # Client settings
  client_id: "kafka_ex",
  sync_timeout: 3000,

  # Consumer group (used for offset storage)
  default_consumer_group: "kafka_ex",

  # Partitioner for produce requests (when partition is nil)
  partitioner: KafkaEx.Producer.Partitioner.Default,

  # Application settings
  disable_default_worker: false,
  max_restarts: 10,
  max_seconds: 60,

  # GenConsumer settings
  commit_interval: 5_000,
  commit_threshold: 100,
  auto_offset_reset: :none

Broker Configuration

Brokers can be configured in several formats:

# List of tuples
brokers: [{"host1", 9092}, {"host2", 9092}]

# CSV string
brokers: "host1:9092,host2:9092"

# Dynamic (MFA or function)
brokers: {MyModule, :get_brokers, []}
brokers: fn -> [...] end

SASL Authentication

config :kafka_ex,
  use_ssl: true,  # Required for SASL
  sasl: %{
    mechanism: :scram,  # :plain or :scram
    username: System.get_env("KAFKA_USER"),
    password: System.get_env("KAFKA_PASS"),
    mechanism_opts: %{algorithm: :sha256}  # For SCRAM
  }

Advanced tuning

Most users never need these. They are surfaced here (and in config/config.exs) so you can see the real defaults without having to grep the source.

• :sleep_for_reconnect (ms, default 400) — delay before retrying a broker reconnect. Applied inside KafkaEx.Client after the socket has died but before the next create_socket/5 attempt. Lower values reconnect faster at the cost of hammering a broker that's down; higher values smooth out flapping brokers at the cost of longer error windows.

• :metadata_update_interval (ms, default 30_000) — periodic metadata refresh cadence. The client does a full Metadata request this often to pick up cluster changes (leader elections, new topics, broker additions/removals). Lower values recover faster from cluster changes; higher values reduce request volume.

• :max_restarts (default 10) — top-level application supervisor restart intensity. If more than :max_restarts children exit in any :max_seconds window, the supervisor shuts down.

• :max_seconds (default 60) — the restart-intensity window that :max_restarts applies to.

Example with non-default values:

config :kafka_ex,
  sleep_for_reconnect: 1_000,       # slower reconnect, less broker hammering
  metadata_update_interval: 10_000, # 3x more frequent metadata refresh
  max_restarts: 3,                  # tighter restart budget
  max_seconds: 30