TantivyEx.Schema (TantivyEx v0.4.1)

View Source

Schema management for TantivyEx.

A schema defines the structure of documents in an index, including field names, types, and indexing options.

Field Types

TantivyEx supports all Tantivy field types:

  • Text fields: Full-text searchable fields with tokenization
  • Numeric fields: u64, i64, f64 for numbers
  • Boolean fields: true/false values
  • Date fields: DateTime values with precision control
  • Facet fields: Hierarchical categorization
  • Bytes fields: Binary data storage
  • JSON fields: Structured JSON object indexing
  • IP Address fields: IPv4 and IPv6 addresses

Field Options

Each field type supports different indexing and storage options:

  • :stored - Field values are stored and retrievable
  • :indexed - Field is searchable (creates inverted index)
  • :fast - Field supports fast field access (like Lucene DocValues)
  • :text - Text field is tokenized and indexed for full-text search
  • :text_stored - Text field is tokenized, indexed, and stored

Fast Fields

Fast fields enable rapid access to field values by document ID, useful for:

  • Sorting and scoring during search
  • Faceted search and aggregations
  • Range queries and filtering
  • Custom collectors that need field access

Summary

Functions

Adds a boolean field to the schema.

Adds a bytes field to the schema.

Adds a date field to the schema.

Adds an f64 (64-bit floating point) field to the schema.

Adds a facet field to the schema.

Adds an i64 (signed 64-bit integer) field to the schema.

Adds an IP address field to the schema.

Adds a JSON object field to the schema.

Adds a text field to the schema.

Adds a text field with a custom tokenizer to the schema.

Adds a u64 (unsigned 64-bit integer) field to the schema.

Checks if a field exists in the schema.

Returns a list of all field names in the schema.

Returns the type of a specific field in the schema.

Creates a new empty schema.

Validates a schema for correctness.

Types

field_options()

@type field_options() :: text_field_options() | numeric_field_options()

numeric_field_options()

@type numeric_field_options() ::
  :indexed | :indexed_stored | :stored | :fast | :fast_stored

t()

@type t() :: reference()

text_field_options()

@type text_field_options() :: :text | :text_stored | :stored

Functions

add_bool_field(schema, field_name, options)

@spec add_bool_field(t(), String.t(), numeric_field_options()) :: t()

Adds a boolean field to the schema.

Useful for flags, status indicators, and binary choices.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_bool_field(schema, "published", :indexed_stored)
iex> is_reference(schema)
true

add_bytes_field(schema, field_name, options)

@spec add_bytes_field(t(), String.t(), numeric_field_options()) :: t()

Adds a bytes field to the schema.

Useful for binary data, hashes, and raw byte sequences.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_bytes_field(schema, "file_hash", :stored)
iex> is_reference(schema)
true

add_date_field(schema, field_name, options)

@spec add_date_field(t(), String.t(), numeric_field_options()) :: t()

Adds a date field to the schema.

Useful for timestamps, publication dates, and temporal data.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_date_field(schema, "created_at", :fast_stored)
iex> is_reference(schema)
true

add_f64_field(schema, field_name, options)

@spec add_f64_field(t(), String.t(), numeric_field_options()) :: t()

Adds an f64 (64-bit floating point) field to the schema.

Useful for prices, ratings, scores, and other decimal values.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_f64_field(schema, "rating", :fast_stored)
iex> is_reference(schema)
true

add_facet_field(schema, field_name)

@spec add_facet_field(t(), String.t()) :: t()

Adds a facet field to the schema.

Facet fields enable hierarchical categorization and faceted search. Facets are always indexed and stored by design.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_facet_field(schema, "category")
iex> is_reference(schema)
true

add_i64_field(schema, field_name, options)

@spec add_i64_field(t(), String.t(), numeric_field_options()) :: t()

Adds an i64 (signed 64-bit integer) field to the schema.

Useful for signed integers, timestamps, and other numeric data.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_i64_field(schema, "timestamp", :fast_stored)
iex> is_reference(schema)
true

add_ip_addr_field(schema, field_name, options)

@spec add_ip_addr_field(t(), String.t(), numeric_field_options()) :: t()

Adds an IP address field to the schema.

Supports both IPv4 and IPv6 addresses for network data indexing.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_ip_addr_field(schema, "client_ip", :indexed_stored)
iex> is_reference(schema)
true

add_json_field(schema, field_name, options)

@spec add_json_field(t(), String.t(), text_field_options()) :: t()

Adds a JSON object field to the schema.

JSON fields allow indexing and searching within structured JSON data.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_json_field(schema, "metadata", :text_stored)
iex> is_reference(schema)
true

add_text_field(schema, field_name, options)

@spec add_text_field(t(), String.t(), text_field_options()) :: t()

Adds a text field to the schema.

Text fields are used for full-text search with tokenization and analysis.

Parameters

  • schema: The schema to modify
  • field_name: The name of the field
  • options: Field indexing and storage options

Field Options

  • :text - Field is tokenized and indexed for full-text search
  • :text_stored - Field is tokenized, indexed, and stored for retrieval
  • :stored - Field is only stored (not searchable)
  • :fast - Field is tokenized, indexed, and optimized for fast access
  • :fast_stored - Field is tokenized, indexed with positions, stored, and optimized for fast access and phrase queries

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_text_field(schema, "title", :text_stored)
iex> schema = TantivyEx.Schema.add_text_field(schema, "body", :text)
iex> is_reference(schema)
true

add_text_field_with_tokenizer(schema, field_name, options, tokenizer)

@spec add_text_field_with_tokenizer(t(), String.t(), text_field_options(), String.t()) ::
  t()

Adds a text field with a custom tokenizer to the schema.

Parameters

  • schema: The schema to modify
  • field_name: The name of the field
  • options: Field indexing and storage options
  • tokenizer: The tokenizer to use ("default", "raw", "en_stem", etc.)

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_text_field_with_tokenizer(schema, "title", :text_stored, "en_stem")
iex> is_reference(schema)
true

add_u64_field(schema, field_name, options)

@spec add_u64_field(t(), String.t(), numeric_field_options()) :: t()

Adds a u64 (unsigned 64-bit integer) field to the schema.

Parameters

  • schema: The schema to modify
  • field_name: The name of the field
  • options: Field indexing and storage options

Field Options

  • :indexed - Field is indexed for fast filtering and range queries
  • :indexed_stored - Field is indexed and stored for retrieval
  • :stored - Field is only stored (not searchable)
  • :fast - Field is stored as a fast field for rapid access
  • :fast_stored - Field is both fast and stored

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_u64_field(schema, "price", :indexed_stored)
iex> schema = TantivyEx.Schema.add_u64_field(schema, "views", :fast)
iex> is_reference(schema)
true

field_exists?(schema, field_name)

@spec field_exists?(t(), String.t()) :: boolean()

Checks if a field exists in the schema.

Parameters

  • schema: The schema to check
  • field_name: The name of the field to check for

Examples

iex> TantivyEx.Schema.field_exists?(schema, "title")
true
iex> TantivyEx.Schema.field_exists?(schema, "nonexistent_field")
false

get_field_names(schema)

@spec get_field_names(t()) :: [String.t()]

Returns a list of all field names in the schema.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_text_field(schema, "title", :text_stored)
iex> schema = TantivyEx.Schema.add_u64_field(schema, "price", :indexed)
iex> TantivyEx.Schema.get_field_names(schema)
["title", "price"]

get_field_type(schema, field_name)

@spec get_field_type(t(), String.t()) :: {:ok, String.t()} | {:error, String.t()}

Returns the type of a specific field in the schema.

Examples

iex> schema = TantivyEx.Schema.new()
iex> schema = TantivyEx.Schema.add_text_field(schema, "title", :text_stored)
iex> TantivyEx.Schema.get_field_type(schema, "title")
{:ok, "text"}

iex> TantivyEx.Schema.get_field_type(schema, "nonexistent")
{:error, "Field 'nonexistent' not found"}

new()

@spec new() :: t()

Creates a new empty schema.

Examples

iex> schema = TantivyEx.Schema.new()
iex> is_reference(schema)
true

validate(schema)

@spec validate(t()) :: {:ok, String.t()} | {:error, String.t()}

Validates a schema for correctness.

Checks for basic requirements like having at least one field.

Examples

iex> schema = TantivyEx.Schema.new()
iex> TantivyEx.Schema.validate(schema)
{:error, "Schema must have at least one field"}

iex> schema = TantivyEx.Schema.add_text_field(schema, "title", :text)
iex> TantivyEx.Schema.validate(schema)
{:ok, "Schema is valid"}