WeaviateEx.API.Quantizer (WeaviateEx v0.7.4)

View Source

Quantizer configuration structs for vector index compression.

Provides typed structs for configuring vector quantization methods:

  • PQConfig - Product Quantization
  • BQConfig - Binary Quantization
  • SQConfig - Scalar Quantization
  • RQConfig - Rotational Quantization

Each struct provides new/1, to_api/1, and from_api/1 functions for creating, serializing, and deserializing configurations.

Usage

# Create a PQ configuration
pq = Quantizer.pq(segments: 128, centroids: 256)

# Use with HNSW index
VectorConfig.hnsw_index(quantizer: Quantizer.to_api(pq))

# Parse from API response
config = Quantizer.from_api(api_response)

Quantization Methods

  • Product Quantization (PQ): Compresses vectors by dividing them into segments and quantizing each segment independently. Good for memory reduction.

  • Binary Quantization (BQ): Converts vectors to binary representations. Most aggressive compression, fastest search, some accuracy loss.

  • Scalar Quantization (SQ): Quantizes each dimension independently. Good balance of compression and accuracy.

  • Rotational Quantization (RQ): Uses rotational transformations before quantization. Advanced method with configurable bit depth.

Summary

Types

quantizer_config()
quantizer_type()

Functions

bq(opts \\ [])

Create a Binary Quantization configuration.

detect_type(api_data)

Detect the quantizer type from an API response map.

from_api(api_data)

Parse a quantizer configuration from an API response.

pq(opts \\ [])

Create a Product Quantization configuration.

rq(opts \\ [])

Create a Rotational Quantization configuration.

sq(opts \\ [])

Create a Scalar Quantization configuration.

to_api(config)

Convert a quantizer config struct to API format.

Types

quantizer_config()

@type quantizer_config() ::
  WeaviateEx.API.Quantizer.PQConfig.t()
  | WeaviateEx.API.Quantizer.BQConfig.t()
  | WeaviateEx.API.Quantizer.SQConfig.t()
  | WeaviateEx.API.Quantizer.RQConfig.t()

quantizer_type()

@type quantizer_type() :: :pq | :bq | :sq | :rq

Functions

bq(opts \\ [])

@spec bq(keyword()) :: WeaviateEx.API.Quantizer.BQConfig.t()

Create a Binary Quantization configuration.

Convenience alias for BQConfig.new/1.

Options

  • :enabled - Enable BQ (default: true)
  • :cache - Enable vector cache
  • :rescore_limit - Number of candidates to rescore

Examples

iex> Quantizer.bq(cache: true)
%BQConfig{enabled: true, cache: true}

detect_type(api_data)

@spec detect_type(map()) :: quantizer_type() | nil

Detect the quantizer type from an API response map.

Returns :pq, :bq, :sq, :rq, or nil if no quantizer is configured.

Examples

iex> Quantizer.detect_type(%{"pq" => %{"enabled" => true}})
:pq

iex> Quantizer.detect_type(%{})
nil

from_api(api_data)

@spec from_api(map()) :: quantizer_config() | nil

Parse a quantizer configuration from an API response.

Automatically detects the quantizer type and returns the appropriate struct. Returns nil if no quantizer is configured.

Examples

iex> Quantizer.from_api(%{"pq" => %{"enabled" => true, "segments" => 128}})
%PQConfig{enabled: true, segments: 128}

iex> Quantizer.from_api(%{})
nil

pq(opts \\ [])

@spec pq(keyword()) :: WeaviateEx.API.Quantizer.PQConfig.t()

Create a Product Quantization configuration.

Convenience alias for PQConfig.new/1.

Options

  • :enabled - Enable PQ (default: true)
  • :training_limit - Number of vectors to train on
  • :segments - Number of segments (0 for auto)
  • :centroids - Number of centroids per segment (default: 256)
  • :encoder - Encoder configuration map with :type and :distribution

Examples

iex> Quantizer.pq(segments: 128, centroids: 256)
%PQConfig{enabled: true, segments: 128, centroids: 256}

rq(opts \\ [])

@spec rq(keyword()) :: WeaviateEx.API.Quantizer.RQConfig.t()

Create a Rotational Quantization configuration.

Convenience alias for RQConfig.new/1.

Options

  • :enabled - Enable RQ (default: true)
  • :bits - Number of bits for quantization (4, 8, or 16)
  • :cache - Enable vector cache
  • :rescore_limit - Number of candidates to rescore
  • :training_limit - Number of vectors to train on

Examples

iex> Quantizer.rq(bits: 8, cache: true)
%RQConfig{enabled: true, bits: 8, cache: true}

sq(opts \\ [])

@spec sq(keyword()) :: WeaviateEx.API.Quantizer.SQConfig.t()

Create a Scalar Quantization configuration.

Convenience alias for SQConfig.new/1.

Options

  • :enabled - Enable SQ (default: true)
  • :cache - Enable vector cache
  • :rescore_limit - Number of candidates to rescore
  • :training_limit - Number of vectors to train on

Examples

iex> Quantizer.sq(training_limit: 50_000)
%SQConfig{enabled: true, training_limit: 50_000}

to_api(config)

@spec to_api(quantizer_config() | nil) :: map()

Convert a quantizer config struct to API format.

Returns an empty map for nil.

Examples

iex> pq = PQConfig.new(segments: 128)
iex> Quantizer.to_api(pq)
%{"pq" => %{"enabled" => true, "segments" => 128}}

iex> Quantizer.to_api(nil)
%{}