TantivyEx.SearchResults (TantivyEx v0.4.1)

View Source

Comprehensive search results processing and formatting for TantivyEx.

This module provides advanced result processing capabilities including:

  • Result formatting and normalization
  • Highlighting and snippets
  • Metadata enhancement
  • Pagination support
  • Performance metrics
  • Result aggregation and faceting
  • Schema-aware field mapping in results

Features

Result Processing

  • Schema-aware field type conversion
  • Automatic highlighting of search terms
  • Snippet extraction with context
  • Score normalization and ranking
  • Metadata enrichment

Performance Optimization

  • Lazy loading of document content
  • Configurable result processing
  • Memory-efficient handling of large result sets
  • Streaming support for bulk operations

Analytics Support

  • Search performance metrics
  • Result quality analysis
  • Click-through tracking preparation
  • Query analysis helpers

Usage Examples

# Basic result processing
{:ok, results} = TantivyEx.Searcher.search(searcher, query, 10)
{:ok, processed} = SearchResults.process(results, schema, options)

# Advanced processing with highlighting
options = [
  highlight: true,
  snippet_length: 200,
  include_metadata: true,
  normalize_scores: true
]
{:ok, enhanced_results} = SearchResults.enhance(results, schema, query, options)

# Pagination with metadata
{:ok, page_results} = SearchResults.paginate(results, page: 2, per_page: 20)

# Result aggregation
{:ok, aggregated} = SearchResults.aggregate(results, [:category, :price_range])

Summary

Functions

Aggregates search results by specified fields to generate facets.

Extracts and analyzes query performance metrics from search results.

Enhances search results with highlighting, snippets, and metadata.

Extracts query terms from a query string or Query object.

Formats search results for API responses with consistent structure.

Paginates search results with metadata about pagination state.

Processes raw search results with schema-aware field mapping and type conversion.

Types

pagination_options()

@type pagination_options() :: [
  page: pos_integer(),
  per_page: pos_integer(),
  include_total: boolean()
]

processed_result()

@type processed_result() :: %{
  score: float(),
  normalized_score: float(),
  doc_id: pos_integer(),
  document: map(),
  highlights: map(),
  snippet: String.t(),
  metadata: map()
}

result_options()

@type result_options() :: [
  highlight: boolean(),
  snippet_length: pos_integer(),
  include_metadata: boolean(),
  normalize_scores: boolean(),
  max_highlights: pos_integer(),
  highlight_tags: {String.t(), String.t()},
  schema_validation: boolean()
]

search_result()

@type search_result() :: %{score: float(), doc_id: pos_integer(), document: map()}

Functions

aggregate(results, facet_fields)

@spec aggregate([map()], [String.t()]) :: {:ok, map()} | {:error, String.t()}

Aggregates search results by specified fields to generate facets.

This is a simple post-processing aggregation for compatibility. For advanced aggregations including metric calculations, histograms, and nested aggregations, use TantivyEx.Aggregation.

Parameters

  • results: List of search results
  • facet_fields: List of field names to aggregate by

Examples

iex> {:ok, facets} = SearchResults.aggregate(results, ["category", "price_range"])
iex> facets
%{
  "category" => %{"books" => 15, "electronics" => 8, "clothing" => 12},
  "price_range" => %{"0-25" => 10, "25-50" => 15, "50+" => 10}
}

For advanced aggregations, use:

# Terms aggregation with Tantivy's native aggregation system
aggregations = %{
  "categories" => TantivyEx.Aggregation.terms("category", size: 10)
}
{:ok, agg_results} = TantivyEx.Aggregation.run(searcher, query, aggregations)

analyze_performance(results, execution_time_ms)

@spec analyze_performance([map()], non_neg_integer()) :: map()

Extracts and analyzes query performance metrics from search results.

Parameters

  • results: Search results
  • execution_time: Search execution time in milliseconds

Examples

iex> metrics = SearchResults.analyze_performance(results, 15)
iex> metrics
%{
  result_count: 25,
  execution_time_ms: 15,
  avg_score: 0.85,
  score_distribution: %{high: 5, medium: 15, low: 5},
  recommendations: ["Consider query optimization for better scores"]
}

enhance(results, schema, query, options \\ [])

@spec enhance(
  [search_result()],
  TantivyEx.Schema.t(),
  TantivyEx.Query.t() | String.t(),
  result_options()
) :: {:ok, [processed_result()]} | {:error, String.t()}

Enhances search results with highlighting, snippets, and metadata.

This function provides advanced result enhancement including search term highlighting, snippet extraction, and metadata enrichment.

Parameters

  • results: List of search results
  • schema: Schema for field processing
  • query: Original search query for highlighting
  • options: Enhancement options

Examples

iex> options = [highlight: true, snippet_length: 150, include_metadata: true]
iex> {:ok, enhanced} = SearchResults.enhance(results, schema, query, options)
iex> enhanced |> hd() |> Map.get("highlights")
%{"title" => ["Introduction to <mark>Elixir</mark>"], "content" => ["Learn <mark>Elixir</mark> programming"]}

extract_query_terms(query, schema)

@spec extract_query_terms(String.t() | term(), term()) ::
  {:ok, [String.t()]} | {:error, String.t()}

Extracts query terms from a query string or Query object.

Parameters

  • query: The query string or Query object to extract terms from
  • schema: The schema to use for term extraction

Returns

  • {:ok, terms} where terms is a list of extracted terms
  • {:error, reason} if extraction fails

Examples

{:ok, terms} = SearchResults.extract_query_terms("hello world", schema)
# Returns: {:ok, ["hello", "world"]}

{:ok, terms} = SearchResults.extract_query_terms(query_object, schema)
# Returns: {:ok, ["extracted", "terms"]}

format_for_api(results, options \\ [])

@spec format_for_api(
  [map()],
  keyword()
) :: {:ok, map()} | {:error, String.t()}

Formats search results for API responses with consistent structure.

Parameters

  • results: Processed search results
  • options: Formatting options

Examples

iex> {:ok, api_response} = SearchResults.format_for_api(results, query: "elixir")
iex> api_response
%{
  "results" => [...],
  "metadata" => %{
    "query" => "elixir",
    "total_count" => 45,
    "execution_time_ms" => 12,
    "max_score" => 1.0
  }
}

paginate(results, options \\ [])

@spec paginate([map()], pagination_options()) :: {:ok, map()} | {:error, String.t()}

Paginates search results with metadata about pagination state.

Parameters

  • results: List of search results
  • options: Pagination options

Examples

iex> {:ok, page_data} = SearchResults.paginate(results, page: 2, per_page: 10)
iex> page_data.items |> length()
10
iex> page_data.metadata
%{current_page: 2, per_page: 10, total_pages: 5, has_next: true, has_prev: true}

process(results, schema, options \\ [])

@spec process([search_result()], TantivyEx.Schema.t(), result_options()) ::
  {:ok, [map()]} | {:error, String.t()}

Processes raw search results with schema-aware field mapping and type conversion.

This is the core result processing function that normalizes field types according to the schema definition and ensures consistent result format.

Parameters

  • results: List of raw search results from Searcher
  • schema: Schema for field type validation and conversion
  • options: Processing options (optional)

Examples

iex> {:ok, results} = TantivyEx.Searcher.search(searcher, query, 10)
iex> {:ok, processed} = SearchResults.process(results, schema)
iex> processed |> hd() |> Map.keys()
["score", "doc_id", "title", "content", "price", "published_at"]