Rag.Reranker.LLM (rag v0.3.4)

View Source

LLM-based reranker that scores documents using a language model.

This reranker uses an LLM to evaluate the relevance of each document to the query and assigns new scores. Documents are then sorted by these LLM-generated relevance scores.

How it works

  1. Formats the query and documents into a prompt
  2. Sends the prompt to an LLM via the Router
  3. Parses the LLM's scoring response (JSON format)
  4. Updates document scores and sorts by relevance
  5. Optionally limits results with top_k

Usage

# With default router (auto-detected providers)
reranker = Rag.Reranker.LLM.new()
{:ok, docs} = Rag.Reranker.rerank(reranker, query, documents)

# With custom router
{:ok, router} = Rag.Router.new(providers: [:gemini])
reranker = Rag.Reranker.LLM.new(router: router)

# With options
{:ok, docs} = Rag.Reranker.rerank(reranker, query, documents,
  top_k: 5,
  normalize_scores: true
)

Options

  • :top_k - Limit to top K documents after reranking
  • :normalize_scores - Normalize scores to 0-1 range (default: false)

Custom Prompt Template

You can provide a custom prompt template with placeholders:

  • {query} - The search query

  • {documents} - Formatted list of documents

    template = """ Rate these documents for the query. Query: {query} Documents: {documents} Return JSON with scores. """

    reranker = Rag.Reranker.LLM.new(prompt_template: template)

Summary

Types

t()

Functions

new(opts \\ [])

Creates a new LLM-based reranker.

rerank(reranker, query, documents, opts)

Reranks documents using LLM-based relevance scoring.

Types

t()

@type t() :: %Rag.Reranker.LLM{prompt_template: String.t(), router: Rag.Router.t()}

Functions

new(opts \\ [])

@spec new(keyword()) :: t()

Creates a new LLM-based reranker.

Options

  • :router - A configured Router struct. If not provided, creates one with auto-detection
  • :prompt_template - Custom prompt template string with {query} and {documents} placeholders

Examples

# Default configuration
reranker = Rag.Reranker.LLM.new()

# With custom router
{:ok, router} = Rag.Router.new(providers: [:gemini, :claude])
reranker = Rag.Reranker.LLM.new(router: router)

# With custom prompt
template = "Score these docs: {query} - {documents}"
reranker = Rag.Reranker.LLM.new(prompt_template: template)

rerank(reranker, query, documents, opts)

@spec rerank(t(), String.t(), [Rag.Reranker.document()], keyword()) ::
  {:ok, [Rag.Reranker.document()]} | {:error, term()}

Reranks documents using LLM-based relevance scoring.

Parameters

  • reranker - The LLM reranker struct
  • query - The search query
  • documents - List of documents to rerank
  • opts - Options:
    • :top_k - Return only top K documents (default: all)
    • :normalize_scores - Normalize scores to 0-1 range (default: false)

Returns

  • {:ok, reranked_documents} - Documents sorted by LLM scores
  • {:error, reason} - If LLM call fails or response is invalid

Examples

reranker = Rag.Reranker.LLM.new()
docs = [
  %{id: 1, content: "Elixir programming", score: 0.7, metadata: %{}},
  %{id: 2, content: "Python basics", score: 0.8, metadata: %{}}
]

{:ok, reranked} = Rag.Reranker.LLM.rerank(
  reranker,
  "What is Elixir?",
  docs,
  top_k: 1
)