View Source API Reference kafka_ex v0.14.0

Kafka API

Handles compression/decompression of messages.

Configuring KafkaEx

A process that manages membership in a Kafka consumer group.

Contains typespecs and reference algorithms for assigning partitions

Default partitioner implementation.

A behaviour module for implementing a Kafka consumer.

A supervisor for managing GenConsumer processes that are part of a consumer group.

Legacy default partitioner implementation.

KafkaEx implementation of Client used to connect to Kafka Broker

Behaviour for any network client. Created mainly to allow mocking request & responses in unit tests.

Kayrock-compatible KafkaEx.Server implementation

This module is responsible for defining the behaviour of a protocol.

This module is used to build request for KafkaEx.New.Client. It's main decision point which protocol to use for building request and what is required version.

This module is used to parse response from KafkaEx.New.Client. It's main decision point which protocol to use for parsing response

This module interfaces Kafka through the New.Client implementation

This module handles Describe Groups request & response parsing. Request is built using Kayrock protocol, response is parsed to native KafkaEx structs.

This protocol is used to build Describe Groups request

This protocol is used to parse Describe Groups response

This module handles Kayrock request & response handling & parsing. Once Kafka Ex v1.0 is released, this module will be renamed to KayrockProtocol and will become a separated package.

Encapsulates what we know about a broker and our connection

Encapsulates what we know about the state of a Kafka broker cluster

Encapsulates what we know about consumer group

Encapsulates what we know about a consumer group member

Encapsulates what we know about a consumer group. The current assignment provided by the group leader (will only be present if the group is stable).

Encapsulates what we know about a consumer group member partition assignment. Will only be present if the group is stable and is assigned to given topic.

Defines node selector functions and macros

Encapsulates what we know about a partition

Encapsulates what we know about a topic

Behaviour definition for partitioners, that assigns partitions for requests.

Implementation of the Kafka ApiVersions request and response APIs

A collection of common request generation and response parsing functions for the Kafka wire protocol.

Implementation of the Kafka ConsumerMetadata request and response APIs

Implementation of the Kafka CreateTopics request and response APIs

Implementation of the Kafka DeleteTopics request and response APIs

Implementation of the Kafka Fetch request and response APIs

Implementation of the Kafka Heartbeat request and response APIs

Implementation of the Kafka JoinGroup request and response APIs

Implementation of the Kafka Heartbeat request and response APIs

Implementation of the Kafka Offset request and response APIs

Implementation of the Kafka OffsetCommit request and response APIs

Implementation of the Kafka OffsetFetch request and response APIs

Implementation of the Kafka Produce request and response APIs

• key: is used for partition assignment, can be nil, when none is provided it is defaulted to nil
• value: is the message to be written to Kafka logs.
• headers: is a list of keys and values that adds metadata to messages
• timestamp: timestamp (kafka_version: "kayrock" ONLY)

• required_acks: indicates how many acknowledgements the servers should receive before responding to the request. If it is 0 the server will not send any response (this is the only case where the server will not reply to a request). If it is 1, the server will wait the data is written to the local log before sending a response. If it is -1 the server will block until the message is committed by all in sync replicas before sending a response. For any number > 1 the server will block waiting for this number of acknowledgements to occur (but the server will never wait for more acknowledgements than there are in-sync replicas), default is 0
• timeout: provides a maximum time in milliseconds the server can await the receipt of the number of acknowledgements in RequiredAcks, default is 100 milliseconds

Implementation of the Kafka SyncGroup request and response APIs

Defines the KafkaEx.Server behavior that all Kafka API servers must implement, this module also provides some common callback functions that are injected into the servers that use it.

Implements KafkaEx.Server behaviors for Kafka 0.10.1 API.

Implements KafkaEx.Server behaviors for Kafka >= 0.8.0 < 0.8.2 API.

Implements KafkaEx.Server behaviors for kafka >= 0.8.2 < 0.9.0 API.

Implements KafkaEx.Server behaviors for Kafka >= 0.9.0 < 0.10.1 API.

This module handle all socket related operations.

Utility module that provides Murmur hashing algorithm.