View Source HNSWLib.Index (hnswlib v0.1.5)

Documentation for HNSWLib.Index.

Summary

Functions

Add items to the index.

Get the current number of elements in the index.

Get the ef parameter.

Get the ef_construction parameter.

Get a list of existing IDs in the index.

Retrieve items from the index using IDs.

Get the M parameter.

Get the maximum number of elements the index can hold.

Get the number of threads to use.

Get the size the of index file.

Query the index with a single vector or a list of vectors.

Mark a label as deleted.

Resize the index.

Save current index to disk.

Set the ef parameter.

Set the number of threads to use.

Unmark a label as deleted.

Functions

Link to this function

add_items(self, data, opts \\ [])

View Source
@spec add_items(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  Nx.Tensor.t(),
  ids: Nx.Tensor.t() | [non_neg_integer()] | nil,
  num_threads: integer(),
  replace_deleted: false
) :: :ok | {:error, String.t()}

Add items to the index.

Positional Parameters
  • data: Nx.Tensor.t().

    Data to add to the index.

Keyword Parameters
  • ids: Nx.Tensor.t() | [non_neg_integer()] | nil.

    IDs to assign to the data.

    If nil, IDs will be assigned sequentially starting from 0.

    Defaults to nil.

  • num_threads: integer().

    Number of threads to use.

    If set to -1, the number of threads will be automatically determined.

    Defaults to -1.

  • replace_deleted: boolean().

    Whether to replace deleted items.

    Defaults to false.

@spec get_current_count(%HNSWLib.Index{dim: term(), reference: term(), space: term()}) ::
  {:ok, integer()} | {:error, String.t()}

Get the current number of elements in the index.

@spec get_ef(%HNSWLib.Index{dim: term(), reference: term(), space: term()}) ::
  {:ok, non_neg_integer()} | {:error, String.t()}

Get the ef parameter.

Link to this function

get_ef_construction(self)

View Source
@spec get_ef_construction(%HNSWLib.Index{
  dim: term(),
  reference: term(),
  space: term()
}) ::
  {:ok, integer()} | {:error, String.t()}

Get the ef_construction parameter.

@spec get_ids_list(%HNSWLib.Index{dim: term(), reference: term(), space: term()}) ::
  {:ok, [integer()]} | {:error, String.t()}

Get a list of existing IDs in the index.

@spec get_items(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  Nx.Tensor.t() | [integer()]
) :: {:ok, [binary()]} | {:error, String.t()}

Retrieve items from the index using IDs.

Positional Parameters
  • ids: Nx.Tensor.t() | [non_neg_integer()].

    IDs to retrieve.

@spec get_m(%HNSWLib.Index{dim: term(), reference: term(), space: term()}) ::
  {:ok, integer()} | {:error, String.t()}

Get the M parameter.

@spec get_max_elements(%HNSWLib.Index{dim: term(), reference: term(), space: term()}) ::
  {:ok, integer()} | {:error, String.t()}

Get the maximum number of elements the index can hold.

@spec get_num_threads(%HNSWLib.Index{dim: term(), reference: term(), space: term()}) ::
  {:ok, integer()} | {:error, String.t()}

Get the number of threads to use.

@spec index_file_size(%HNSWLib.Index{dim: term(), reference: term(), space: term()}) ::
  {:ok, non_neg_integer()} | {:error, String.t()}

Get the size the of index file.

Link to this function

knn_query(self, query, opts \\ [])

View Source
@spec knn_query(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  Nx.Tensor.t() | binary() | [binary()],
  k: pos_integer(),
  num_threads: integer()
) :: {:ok, Nx.Tensor.t(), Nx.Tensor.t()} | {:error, String.t()}

Query the index with a single vector or a list of vectors.

Positional Parameters
  • query: Nx.Tensor.t() | binary() | [binary()].

    A vector or a list of vectors to query.

    If query is a list of vectors, the vectors must be of the same dimension.

Keyword Paramters
  • k: pos_integer().

    Number of nearest neighbors to return.

  • num_threads: integer().

    Number of threads to use.

Link to this function

load_index(space, dim, path, opts \\ [])

View Source
@spec load_index(:cosine | :ip | :l2, non_neg_integer(), Path.t(),
  max_elements: non_neg_integer(),
  allow_replace_deleted: boolean()
) ::
  {:ok, %HNSWLib.Index{dim: term(), reference: term(), space: term()}}
  | {:error, String.t()}

Load index from disk.

Positional Parameters
  • space: :cosine | :ip | :l2.

    An atom that indicates the vector space. Valid values are

    • :cosine, cosine space
    • :ip, inner product space
    • :l2, L2 space
  • dim: non_neg_integer().

    Number of dimensions for each vector.

  • path: Path.t().

    Path to load the index from.

Keyword Parameters
  • max_elements: non_neg_integer().

    Maximum number of elements to load from the index. If set to 0, all elements will be loaded. Default: 0.

  • allow_replace_deleted: boolean().

Link to this function

mark_deleted(self, label)

View Source
@spec mark_deleted(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  non_neg_integer()
) ::
  :ok | {:error, String.t()}

Mark a label as deleted.

Positional Parameters
  • label: non_neg_integer().

    Label to mark as deleted.

Link to this function

new(space, dim, max_elements, opts \\ [])

View Source
@spec new(:cosine | :ip | :l2, non_neg_integer(), pos_integer(),
  m: non_neg_integer(),
  ef_construction: non_neg_integer(),
  random_seed: non_neg_integer(),
  allow_replace_deleted: boolean()
) ::
  {:ok, %HNSWLib.Index{dim: term(), reference: term(), space: term()}}
  | {:error, String.t()}

Construct a new Index

Positional Parameters
  • space: :cosine | :ip | :l2.

    An atom that indicates the vector space. Valid values are

    • :cosine, cosine space
    • :ip, inner product space
    • :l2, L2 space
  • dim: non_neg_integer().

    Number of dimensions for each vector.

  • max_elements: pos_integer().

    Number of maximum elements.

Keyword Paramters
  • m: non_neg_integer().

    M is tightly connected with internal dimensionality of the data strongly affects the memory consumption

  • ef_construction: non_neg_integer().

    controls index search speed/build speed tradeoff

  • random_seed: non_neg_integer().

  • allow_replace_deleted: boolean().

Link to this function

resize_index(self, new_size)

View Source
@spec resize_index(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  non_neg_integer()
) ::
  :ok | {:error, String.t()}

Resize the index.

Positional Parameters
  • new_size: non_neg_integer().

    New size of the index.

@spec save_index(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  Path.t()
) :: :ok

Save current index to disk.

Positional Parameters
  • path: Path.t().

    Path to save the index to.

@spec set_ef(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  non_neg_integer()
) ::
  :ok | {:error, String.t()}

Set the ef parameter.

Link to this function

set_num_threads(self, new_num_threads)

View Source
@spec set_num_threads(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  integer()
) ::
  {:ok, integer()} | {:error, String.t()}

Set the number of threads to use.

Link to this function

unmark_deleted(self, label)

View Source
@spec unmark_deleted(
  %HNSWLib.Index{dim: term(), reference: term(), space: term()},
  non_neg_integer()
) ::
  :ok | {:error, String.t()}

Unmark a label as deleted.

Positional Parameters
  • label: non_neg_integer().

    Label to unmark as deleted.