Milvex.Index (milvex v0.10.2)

Copy Markdown

Builder for Milvus index configurations.

Provides a fluent API for constructing index definitions with validation. Supports vector indexes, sparse indexes, and scalar field indexes.

Vector Index Examples

index = Index.new("embedding", :hnsw, :cosine)
        |> Index.name("my_index")
        |> Index.params(%{M: 16, efConstruction: 256})

Index.flat("embedding", :l2)
Index.ivf_flat("embedding", :ip, nlist: 1024)
Index.hnsw("embedding", :cosine, m: 16, ef_construction: 256)
Index.autoindex("embedding", :l2)

Scalar Index Examples

Index.inverted("category")
Index.stl_sort("price")
Index.trie("username")
Index.bitmap("status")

Summary

Functions

Creates a BITMAP index for low-cardinality scalar fields.

Creates a FLAT index (brute-force search).

Returns list of all supported index types.

Creates an INVERTED index for scalar fields.

Returns list of all supported metric types.

Sets the index name.

Creates a new scalar index configuration.

Creates a new vector index configuration.

Sets or merges additional index parameters.

Returns list of scalar index types.

Creates a SPARSE_INVERTED_INDEX for BM25 full-text search.

Creates an STL_SORT index for numeric scalar fields.

Converts the index configuration to protobuf extra_params.

Creates a Trie index for VARCHAR fields.

Validates the index configuration.

Validates the index and raises on error.

Returns list of vector index types.

Types

index_type()

@type index_type() :: vector_index_type() | scalar_index_type()

inverted_index_algo()

@type inverted_index_algo() :: :daat_maxscore | :daat_wand | :taat_naive

metric_type()

@type metric_type() ::
  :l2
  | :ip
  | :cosine
  | :hamming
  | :jaccard
  | :max_sim_cosine
  | :max_sim_ip
  | :bm25

scalar_index_type()

@type scalar_index_type() :: :inverted | :stl_sort | :trie | :bitmap

t()

@type t() :: %Milvex.Index{
  field_name: String.t(),
  index_type: index_type(),
  metric_type: metric_type() | nil,
  name: String.t() | nil,
  params: map()
}

vector_index_type()

@type vector_index_type() ::
  :flat
  | :ivf_flat
  | :ivf_sq8
  | :ivf_pq
  | :hnsw
  | :autoindex
  | :diskann
  | :gpu_ivf_flat
  | :gpu_ivf_pq
  | :scann
  | :sparse_inverted_index

Functions

autoindex(field_name, metric_type, opts \\ [])

@spec autoindex(String.t(), metric_type(), keyword()) :: t()

Creates an AUTOINDEX.

Lets Milvus automatically choose the best index type and parameters based on data characteristics.

Options

  • :name - Index name (optional)

Examples

Index.autoindex("embedding", :l2)

bitmap(field_name, opts \\ [])

@spec bitmap(
  String.t(),
  keyword()
) :: t()

Creates a BITMAP index for low-cardinality scalar fields.

Supports BOOL, INT8, INT16, INT32, INT64, VARCHAR, and ARRAY fields. Not supported for FLOAT, DOUBLE, or JSON types.

Options

  • :name - Index name (optional)

Examples

Index.bitmap("status")
Index.bitmap("is_active", name: "active_bitmap_idx")

diskann(field_name, metric_type, opts \\ [])

@spec diskann(String.t(), metric_type(), keyword()) :: t()

Creates a DiskANN index.

Disk-based ANN index for very large datasets that don't fit in memory.

Options

  • :name - Index name (optional)

Examples

Index.diskann("embedding", :l2)

flat(field_name, metric_type)

@spec flat(String.t(), metric_type()) :: t()

Creates a FLAT index (brute-force search).

FLAT provides 100% recall but is slower for large datasets. Best for small datasets or when perfect accuracy is required.

Examples

Index.flat("embedding", :l2)
Index.flat("embedding", :cosine)

hnsw(field_name, metric_type, opts \\ [])

@spec hnsw(String.t(), metric_type(), keyword()) :: t()

Creates an HNSW index.

Hierarchical Navigable Small World graph. Excellent performance for high-dimensional vectors with good recall.

Options

  • :m - Maximum number of connections per node (default: 16)
  • :ef_construction - Search depth during index building (default: 256)
  • :name - Index name (optional)

Examples

Index.hnsw("embedding", :cosine)
Index.hnsw("embedding", :l2, m: 32, ef_construction: 512)

index_types()

@spec index_types() :: [index_type()]

Returns list of all supported index types.

inverted(field_name, opts \\ [])

@spec inverted(
  String.t(),
  keyword()
) :: t()

Creates an INVERTED index for scalar fields.

Supports all scalar field types: BOOL, INT8, INT16, INT32, INT64, FLOAT, DOUBLE, VARCHAR, JSON, and ARRAY.

Options

  • :name - Index name (optional)

Examples

Index.inverted("category")
Index.inverted("category", name: "category_idx")

ivf_flat(field_name, metric_type, opts \\ [])

@spec ivf_flat(String.t(), metric_type(), keyword()) :: t()

Creates an IVF_FLAT index.

Inverted File index with flat quantization. Good balance of speed and accuracy for medium-sized datasets.

Options

  • :nlist - Number of cluster units (default: 1024)
  • :name - Index name (optional)

Examples

Index.ivf_flat("embedding", :l2)
Index.ivf_flat("embedding", :ip, nlist: 2048)

ivf_pq(field_name, metric_type, opts \\ [])

@spec ivf_pq(String.t(), metric_type(), keyword()) :: t()

Creates an IVF_PQ index.

IVF with product quantization. Very memory-efficient but with lower accuracy. Best for very large datasets.

Options

  • :nlist - Number of cluster units (default: 1024)
  • :m - Number of subquantizers (default: 8)
  • :nbits - Bits per subquantizer (default: 8)
  • :name - Index name (optional)

Examples

Index.ivf_pq("embedding", :l2)
Index.ivf_pq("embedding", :ip, nlist: 2048, m: 16)

ivf_sq8(field_name, metric_type, opts \\ [])

@spec ivf_sq8(String.t(), metric_type(), keyword()) :: t()

Creates an IVF_SQ8 index.

IVF with scalar quantization. More memory-efficient than IVF_FLAT with slight accuracy trade-off.

Options

  • :nlist - Number of cluster units (default: 1024)
  • :name - Index name (optional)

Examples

Index.ivf_sq8("embedding", :l2)
Index.ivf_sq8("embedding", :ip, nlist: 2048)

metric_types()

@spec metric_types() :: [metric_type()]

Returns list of all supported metric types.

name(index, name)

@spec name(t(), String.t()) :: t()

Sets the index name.

If not set, Milvus will auto-generate a name.

new(field_name, index_type)

@spec new(String.t(), scalar_index_type()) :: t()

Creates a new scalar index configuration.

Scalar indexes do not require a metric type.

Parameters

  • field_name - Name of the scalar field to index
  • index_type - Type of scalar index (:inverted, :stl_sort, :trie, :bitmap)

Examples

Index.new("category", :inverted)
Index.new("age", :stl_sort)
Index.new("name", :trie)
Index.new("status", :bitmap)

new(field_name, index_type, metric_type)

@spec new(String.t(), index_type(), metric_type()) :: t()

Creates a new vector index configuration.

Parameters

  • field_name - Name of the vector field to index
  • index_type - Type of index to create
  • metric_type - Distance metric to use

Examples

Index.new("embedding", :hnsw, :cosine)
Index.new("vectors", :ivf_flat, :l2)

params(index, params)

@spec params(t(), map()) :: t()

Sets or merges additional index parameters.

Parameters are index-type specific. Common parameters include:

  • IVF indexes: nlist (number of cluster units)
  • HNSW: M (max connections), efConstruction (search depth during build)
  • IVF_PQ: m (number of subquantizers), nbits (bits per subquantizer)

scalar_index_types()

@spec scalar_index_types() :: [scalar_index_type()]

Returns list of scalar index types.

scann(field_name, metric_type, opts \\ [])

@spec scann(String.t(), metric_type(), keyword()) :: t()

Creates a SCANN index.

Google's ScaNN (Scalable Nearest Neighbors) implementation. Good balance of speed and accuracy.

Options

  • :nlist - Number of cluster units (default: 1024)
  • :name - Index name (optional)

Examples

Index.scann("embedding", :l2)
Index.scann("embedding", :cosine, nlist: 2048)

sparse_bm25(field_name, opts \\ [])

@spec sparse_bm25(
  String.t(),
  keyword()
) :: t()

Creates a SPARSE_INVERTED_INDEX for BM25 full-text search.

Used for SPARSE_FLOAT_VECTOR fields that receive BM25 function output. Supports inverted index algorithms optimized for sparse vectors.

Options

  • :inverted_index_algo - Algorithm to use (default: :daat_maxscore)
    • :daat_maxscore - Document-at-a-time with MaxScore optimization
    • :daat_wand - Document-at-a-time with WAND optimization
    • :taat_naive - Term-at-a-time naive approach
  • :bm25_k1 - BM25 k1 parameter (default: 1.2)
  • :bm25_b - BM25 b parameter (default: 0.75)
  • :drop_ratio_build - Drop ratio during index build (default: 0.2)
  • :name - Index name (optional)

Examples

Index.sparse_bm25("text_sparse")
Index.sparse_bm25("text_sparse", inverted_index_algo: :daat_wand)
Index.sparse_bm25("text_sparse", bm25_k1: 1.5, bm25_b: 0.8)
Index.sparse_bm25("text_sparse", drop_ratio_build: 0.1)

stl_sort(field_name, opts \\ [])

@spec stl_sort(
  String.t(),
  keyword()
) :: t()

Creates an STL_SORT index for numeric scalar fields.

Supports numeric types: INT8, INT16, INT32, INT64, FLOAT, DOUBLE.

Options

  • :name - Index name (optional)

Examples

Index.stl_sort("price")
Index.stl_sort("timestamp", name: "ts_idx")

to_extra_params(index)

@spec to_extra_params(t()) :: [Milvex.Milvus.Proto.Common.KeyValuePair.t()]

Converts the index configuration to protobuf extra_params.

Returns a list of KeyValuePair structs for use in CreateIndexRequest.

trie(field_name, opts \\ [])

@spec trie(
  String.t(),
  keyword()
) :: t()

Creates a Trie index for VARCHAR fields.

Optimized for prefix matching queries on string fields.

Options

  • :name - Index name (optional)

Examples

Index.trie("username")
Index.trie("email", name: "email_trie_idx")

validate(index)

@spec validate(t()) :: {:ok, t()} | {:error, Milvex.Error.t()}

Validates the index configuration.

Returns {:ok, index} if valid, {:error, error} otherwise.

validate!(index)

@spec validate!(t()) :: t()

Validates the index and raises on error.

vector_index_types()

@spec vector_index_types() :: [vector_index_type()]

Returns list of vector index types.