ExFinalFusion.Native (ex_final_fusion v0.1.2)

@type read_type() ::
  :floret_text
  | :embeddings
  | :mmap_embeddings
  | :fasttext
  | :fasttext_lossy
  | :text
  | :text_lossy
  | :text_dims
  | :text_dims_lossy
  | :word2vec_binary
  | :word2vec_binary_lossy
  | :fifu
  | :word2vec
  | :floret

It allows to specify what function will be used to parse the embeddings file. You can find more in the rust crate documentation

search_options()

@type search_options() :: [
  limit: integer(),
  batch_size: integer(),
  similarity_type: similarity_type(),
  skip: [String.t()]
]

Options passed to the functions that search for embeddings

default options: limit: 1, batch_size: None, means all at once - this is memory intensive. similarity_type: :cosine_similarity, skip: [], only for embedding_similarity as a mask

similarity_type()

@type similarity_type() ::
  :cosine_similarity
  | :angular_similarity
  | :euclidean_similarity
  | :euclidean_distance

Specifies how to calculate the similarity type when returning similarities. This only changes the returned value. Cosine similarity is always used.

Functions

analogy(arg1, arg2, arg3, arg4, arg5 \\ [])

@spec analogy(reference(), String.t(), String.t(), String.t(), Keyword.t()) ::
  {:ok, [{String.t(), float()}]}

returns the calculated analogy

This method returns words that are close in vector space for the analogy query word1 is to word2 as word3 is to ?. More concretely, it searches embeddings that are similar to:

analogy_masked(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8 \\ [])

@spec analogy_masked(
  reference(),
  String.t(),
  bool(),
  String.t(),
  bool(),
  String.t(),
  bool(),
  Keyword.t()
) :: {:ok, [{String.t(), float()}]}

similar to the analogy query but allows to remove queried words from results

dims(arg1)

@spec dims(reference()) :: [integer()]

embedding(arg1, arg2)

@spec embedding(reference(), String.t()) :: {:ok, [float()]}

returns embedding of a word

embedding_batch(arg1, arg2)

@spec embedding_batch(reference(), [String.t()]) :: {:ok, [[float()]]}

returns a list of embeddings for provided word

embedding_similarity(arg1, arg2, arg3 \\ [])

@spec embedding_similarity(reference(), [float()], Keyword.t()) ::
  {:ok, [{String.t(), float()}]}

returns words that are similar to the query vector.

idx(arg1, arg2)

@spec idx(reference(), String.t()) ::
  nil | {:word, [integer()]} | {:subword, [integer()]}

returns the index of a word

len(arg1)

@spec len(reference()) :: integer()

mean_embedding_batch(arg1, arg2)

@spec mean_embedding_batch(reference(), [String.t()]) :: {:ok, [[float()]], float()}

Returns the average embedding for the provided word and the fraction of how many words were included in the calculation.

metadata(arg1)

@spec metadata(reference()) :: map() | nil

returns the metadata as a map or nil

read(arg1, arg2)

@spec read(String.t(), read_type()) :: reference()

Functions available on embeddings module

:floret_text,
:embeddings,
:mmap_embeddings,
:fasttext,
:fasttext_lossy,
:text,
:text_lossy,
:text_dims,
:text_dims_lossy,
:word2vec_binary,
:word2vec_binary_lossy,

Aliases

:fifu = :embeddings,
:word2vec = :word2vec_binary,
:floret = :floret_text

vocab_len(arg1)

@spec vocab_len(reference()) :: integer()

word_similarity(arg1, arg2, arg3 \\ [])

@spec word_similarity(reference(), String.t(), Keyword.t()) ::
  {:ok, [{String.t(), float()}]}

returns words that are similar to the query word.

words(arg1)

@spec words(reference()) :: [String.t()]

returns a list of words included in the embeddings

words_len(arg1)

@spec words_len(reference()) :: integer()