Kaffe.Subscriber (Kaffe v1.28.0)

View Source

Consume messages from a single partition of a single Kafka topic.

Assignments are received from a group consumer member, Kaffe.GroupMember.

Messages are delegated to Kaffe.Worker. The worker is expected to cast back a response, at which time the stored offset will be acked back to Kafka.

The options (ops) to subscribe/7 may include the beginning offset using :begin_offset.

The subscriber reads the following options out of the configuration:

  • max_bytes - The maximum number of message bytes to receive in a batch
  • min_bytes - The minimum number of message bytes to receive in a batch
  • max_wait_time - Maximum number of milliseconds broker will wait for :min_bytes of messages to be collected
  • offset_reset_policy - The native auto.offset.reset option, either :reset_to_earliest or :reset_to_latest.

See: https://github.com/klarna/brucke/blob/master/src/brucke_member.erl Also: https://github.com/klarna/brod/blob/master/src/brod_consumer.erl

Summary

Functions

commit_offsets(subscriber_pid, topic, partition, generation_id, offset)

kafka_message(args \\ [])

(macro)

kafka_message(record, args)

(macro)

kafka_message_set(args \\ [])

(macro)

kafka_message_set(record, args)

(macro)

request_more_messages(subscriber_pid, offset)

stop(subscriber_pid)

subscribe(subscriber_name, group_coordinator_pid, worker_pid, gen_id, topic, partition, ops, config)