ExNlp.Ranking.ProcessingOptions (ex_nlp v0.1.0)

View Source

Shared processing options for ranking algorithms.

These options control how documents and queries are preprocessed before ranking calculations.

Summary

Types

t()

Functions

get_tokenizer(processing_options)

Gets the tokenizer function, using default if not specified.

new(opts \\ [])

Creates processing options from keyword list.

Types

t()

@type t() :: %ExNlp.Ranking.ProcessingOptions{
  language: atom(),
  remove_stopwords: boolean(),
  stem: boolean(),
  stopword_language: atom(),
  tokenizer: (String.t() -> [String.t()]) | nil
}

Functions

get_tokenizer(processing_options)

@spec get_tokenizer(t()) :: (String.t() -> [String.t()])

Gets the tokenizer function, using default if not specified.

new(opts \\ [])

@spec new(keyword()) :: t()

Creates processing options from keyword list.

Examples

iex> opts = ExNlp.Ranking.ProcessingOptions.new(stem: true, language: :french)
iex> opts.stem
true
iex> opts.language
:french