GorillaStream - High-performance time series compression using the Gorilla algorithm.
This library provides lossless compression for time series data using Facebook's Gorilla compression algorithm, optimized for time-stamped floating-point data.
Quick Start
# Sample time series data: {timestamp, value} tuples
data = [
{1609459200, 23.5},
{1609459260, 23.7},
{1609459320, 23.4}
]
# Compress the data
{:ok, compressed} = GorillaStream.compress(data)
# Decompress back to original
{:ok, decompressed} = GorillaStream.decompress(compressed)Key Features
- Lossless Compression: Perfect reconstruction of original data
- High Performance: 1.7M+ points/sec encoding, up to 2M points/sec decoding
- Excellent Compression Ratios: 2-42x compression depending on data patterns
- Production Ready: Comprehensive error handling and validation
- Memory Efficient: ~117 bytes/point memory usage for large datasets
Main Functions
The primary compression functions are provided by GorillaStream.Compression.Gorilla:
GorillaStream.Compression.Gorilla.compress/2- Compress time series dataGorillaStream.Compression.Gorilla.decompress/2- Decompress data
For convenience, this module also provides direct access to these functions.
Summary
Functions
Compresses time series data using the Gorilla algorithm.
Compresses data using a pre-trained zstd dictionary.
Decompresses Gorilla-compressed data back to original format.
Decompresses data using a pre-trained zstd dictionary.
Checks if zstd compression is available.
Functions
Compresses time series data using the Gorilla algorithm.
This is a convenience function that delegates to GorillaStream.Compression.Gorilla.compress/2.
VictoriaMetrics-style preprocessing is ENABLED by default.
Parameters
data- List of{timestamp, value}tuples- Second argument may be either:
zlib_compression?(boolean) to toggle zlib compression (default: false), OR- keyword options, supporting:
:victoria_metrics(boolean, default: true):is_counter(boolean, default: false):scale_decimals(:auto | integer, default: :auto):compression(:none|:zlib|:zstd|:auto, default: :none):compression_level(integer 1-22, zstd only, default: ezstd default):zlib(boolean, default: false) - legacy option, use:compressioninstead
Returns
{:ok, compressed_binary}- Success with compressed data{:error, reason}- Error with description
Examples
iex> data = [{1609459200, 23.5}, {1609459201, 23.7}]
iex> {:ok, compressed} = GorillaStream.compress(data)
iex> is_binary(compressed)
trueCompresses data using a pre-trained zstd dictionary.
See GorillaStream.Compression.Container.compress_with_dict/2.
Decompresses Gorilla-compressed data back to original format.
This is a convenience function that delegates to GorillaStream.Compression.Gorilla.decompress/2.
Parameters
compressed_data- Binary data from compress/2- Second argument may be either:
zlib_compression?(boolean) indicating if zlib was used (default: false), OR- keyword options, supporting:
:compression(:none|:zlib|:zstd|:auto, default: :none):compression_level(integer 1-22, zstd only, default: ezstd default):zlib(boolean, default: false) - legacy option, use:compressioninstead
Returns
{:ok, decompressed_data}- List of{timestamp, value}tuples{:error, reason}- Error with description
Examples
iex> data = [{1609459200, 23.5}, {1609459201, 23.7}]
iex> {:ok, compressed} = GorillaStream.compress(data)
iex> {:ok, decompressed} = GorillaStream.decompress(compressed)
iex> decompressed == data
trueDecompresses data using a pre-trained zstd dictionary.
See GorillaStream.Compression.Container.decompress_with_dict/2.
Checks if zstd compression is available.
Zstd requires the optional ezstd package to be installed.
Examples
iex> GorillaStream.zstd_available?()
true # or false, depending on whether ezstd is installed