Vettore v0.2.3

View Source Vettore (Vettore v0.2.3)

The Vettore library is designed for fast, in-memory operations on vector (embedding) data.

All vectors (embeddings) are stored in a Rust data structure (a HashMap), accessed via a shared resource (using Rustler’s ResourceArc with a Mutex). Core operations include:

- Creating a collection :
  A named set of embeddings with a fixed dimension and a chosen similarity metric (:cosine, :euclidean, :dot,
  :hnsw, :binary).

- Inserting an embedding :
  Add a new embedding (with ID, vector, and optional metadata) to a specific collection.

- Retrieving embeddings :
  Fetch all embeddings from a collection or look up a single embedding by its unique ID.

- Similarity search :
  Given a query vector, calculate a “score” for every embedding in the collection and return the top‑k results
  (e.g. the smallest distances or largest similarities).

# Usage Example

  db = Vettore.new()
  :ok = Vettore.create_collection(db, "my_collection", 3, :euclidean)

  # Insert an embedding via struct:
  embedding = %Vettore.Embedding{value: "my_id or text", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}
  :ok = Vettore.insert(db, "my_collection", embedding)

  # Retrieve it back:
  {:ok, returned_emb} = Vettore.get_by_value(db, "my_collection", "my_id")
  IO.inspect(returned_emb.vector, label: "Retrieved vector")

  # Perform a similarity search:
  {:ok, top_results} = Vettore.similarity_search(db, "my_collection", [1.5, 1.5, 1.5], 2)
  IO.inspect(top_results, label: "Top K search results")

Link to this section Summary

Functions

batch(db, col, embs)

Batch‑insert a list of embeddings. Reject elements that are not %Vettore.Embedding{}. Batch is faster than insert/3 for a large number of embeddings as it avoids to validate vector list on each embedding.

batch!(db, col, embs)

Batch‑insert a list of embeddings. Reject elements that are not %Vettore.Embedding{}. Raise error if the metadata is invalid. Otherwise, return the list of inserted as in batch/3.

create_collection(db, name, dim, dist, opts \\ [])

Create a collection.

delete(db, col, id)

Delete a single embedding.

delete_collection(db, name)

Delete a collection.

get_all(db, col)

Return all embeddings in raw form ({value, vector, metadata} tuples).

get_by_value(db, col, val)

Fetch a single embedding by value (ID) and return it as %Vettore.Embedding{}.

get_by_vector(db, col, vector)

Fetch a single embedding by vector and return it as %Vettore.Embedding{}.

insert(db, col, arg3)

Insert one %Vettore.Embedding{} into the collection. Returns {:ok, value} on success or {:error, reason}.

insert!(db, col, arg3)

Insert! one %Vettore.Embedding{} into the collection. Raise error if the vector is not numeric or the metadata is invalid. Otherwise, return the inserted value.

new()

Allocate an empty in‑memory DB (owned by Rust). Keep the returned reference around – every other call expects it.

rerank(db, col, initial, opts \\ [])

Re‑rank an existing result list with Maximal Marginal Relevance.

similarity_search(db, col, query, opts \\ [])

Similarity / nearest‑neighbour search.

Link to this section Functions

Link to this function

batch(db, col, embs)

View Source 
@spec batch(reference(), String.t(), [Vettore.Embedding.t()]) ::
  {:ok, [String.t()]} | {:error, String.t()}

Batch‑insert a list of embeddings. Reject elements that are not %Vettore.Embedding{}. Batch is faster than insert/3 for a large number of embeddings as it avoids to validate vector list on each embedding.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.batch("my_collection", [%Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}])
{:ok, ["my_id"]}
Link to this function

batch!(db, col, embs)

View Source 
@spec batch!(reference(), String.t(), [Vettore.Embedding.t()]) ::
  {:ok, [String.t()]} | {:error, String.t()}

Batch‑insert a list of embeddings. Reject elements that are not %Vettore.Embedding{}. Raise error if the metadata is invalid. Otherwise, return the list of inserted as in batch/3.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.batch("my_collection", [%Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => [1, 2, 4]}}])
ArgumentError[…]
Link to this function

create_collection(db, name, dim, dist, opts \\ [])

View Source 
@spec create_collection(
  reference(),
  String.t(),
  pos_integer(),
  atom(),
  keyword()
) :: {:ok, String.t()} | {:error, String.t()}

Create a collection.

  • distance must be one of the atoms: :euclidean, :cosine, :dot, :hnsw, or :binary.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean)
{:ok, "my_collection"}
Link to this function

delete(db, col, id)

View Source 
@spec delete(reference(), String.t(), String.t()) ::
  {:ok, String.t()} | {:error, String.t()}

Delete a single embedding.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.delete("my_collection", "my_id")
{:ok, "my_id"}
Link to this function

delete_collection(db, name)

View Source 
@spec delete_collection(reference(), String.t()) ::
  {:ok, String.t()} | {:error, String.t()}

Delete a collection.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.delete_collection("my_collection")
{:ok, "my_collection"}
Link to this function

get_all(db, col)

View Source 
@spec get_all(reference(), String.t()) ::
  {:ok, [{String.t(), [number()], map() | nil}]} | {:error, String.t()}

Return all embeddings in raw form ({value, vector, metadata} tuples).

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.get_all("my_collection")
{:ok, [{"my_id", [1.0, 2.0, 3.0], %{"note" => "hello"}}]}
Link to this function

get_by_value(db, col, val)

View Source 
@spec get_by_value(reference(), String.t(), String.t()) ::
  {:ok, Vettore.Embedding.t()} | {:error, String.t()}

Fetch a single embedding by value (ID) and return it as %Vettore.Embedding{}.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.get_by_value("my_collection", "my_id")
{:ok, %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}}
Link to this function

get_by_vector(db, col, vector)

View Source 
@spec get_by_vector(reference(), String.t(), [number()]) ::
  {:ok, Vettore.Embedding.t()} | {:error, String.t()}

Fetch a single embedding by vector and return it as %Vettore.Embedding{}.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.get_by_vector("my_collection", [1.0, 2.0, 3.0])
{:ok, %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}}
Link to this function

insert(db, col, arg3)

View Source 
@spec insert(reference(), String.t(), Vettore.Embedding.t()) ::
  {:ok, String.t()} | {:error, String.t()}

Insert one %Vettore.Embedding{} into the collection. Returns {:ok, value} on success or {:error, reason}.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}})
{:ok, "my_id"}
Link to this function

insert!(db, col, arg3)

View Source 
@spec insert!(reference(), String.t(), Vettore.Embedding.t()) ::
  {:ok, String.t()} | {:error, String.t()}

Insert! one %Vettore.Embedding{} into the collection. Raise error if the vector is not numeric or the metadata is invalid. Otherwise, return the inserted value.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert!("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => %{"hello" => "world"}}})
ArgumentError[…]
Link to this function

new()

View Source 
@spec new() :: reference()

Allocate an empty in‑memory DB (owned by Rust). Keep the returned reference around – every other call expects it.

Link to this function

rerank(db, col, initial, opts \\ [])

View Source 
@spec rerank(reference(), String.t(), [{String.t(), number()}], keyword()) ::
  {:ok, [{String.t(), number()}]} | {:error, String.t()}

Re‑rank an existing result list with Maximal Marginal Relevance.

Options:

  • :limit – desired output length (default 10)
  • :alpha – relevance‑diversity balance 0.0..1.0 (default 0.5)

# Examples

 iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id2", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id3", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.rerank("my_collection", [{"my_id", 0.0}, {"my_id2", 0.0}, {"my_id3", 0.0}], limit: 1)
 {:ok, [{"my_id", 0.0}]}
Link to this function

similarity_search(db, col, query, opts \\ [])

View Source 
@spec similarity_search(reference(), String.t(), [number()], keyword()) ::
  {:ok, [{String.t(), float()}]} | {:error, String.t()}

Similarity / nearest‑neighbour search.

Options:

  • :limit – number of results (default 10)
  • :filter – metadata map; only embeddings whose metadata contains all key‑value pairs are considered.

Examples

iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.similarity_search("my_collection", [1.0, 2.0, 3.0], limit: 1)
{:ok, [{"my_id", 0.0}]}