View Source Kayrock.Compression (kayrock v0.2.0)

Handles compression/decompression of messages.

NOTE this is a copy of KafkaEx.Compression: https://github.com/kafkaex/kafka_ex/blob/master/lib/kafka_ex/compression.ex

It is duplicated here to avoid creating a circular dependency.

See https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Compression

To add new compression types:

  1. Add the appropriate dependency to mix.exs (don't forget to add it to the application list).
  2. Add the appropriate attribute value and compression_type atom.
  3. Add a decompress function clause.
  4. Add a compress function clause.

Summary

Functions

This function should pattern match on the compression_type atom and return the compressed data as well as the corresponding attribute value.

This function should pattern match on the attribute value and return the decompressed data.

Types

@type attribute_t() :: integer()
@type compression_type_t() :: :snappy | :gzip

Functions

@spec compress(compression_type_t(), iodata()) :: {binary(), attribute_t()}

This function should pattern match on the compression_type atom and return the compressed data as well as the corresponding attribute value.

@spec decompress(attribute_t(), binary()) :: binary()

This function should pattern match on the attribute value and return the decompressed data.

Link to this function

snappy_decompress_chunk(arg, so_far)

View Source