Configuration

Encoder Options

{:ok, encoder} = Stephen.Encoder.load(
  model: "colbert-ir/colbertv2.0",
  max_query_length: 32,
  max_doc_length: 180,
  projection_dim: 128
)

Model Selection

Parameters

ColBERT vs Standard Models

When loading official ColBERT models, Stephen:

1. Downloads the model from HuggingFace
2. Loads BERT backbone via Bumblebee
3. Extracts trained projection weights from SafeTensors

For other models, Stephen initializes a random projection matrix.

Document Encoding Options

embeddings = Stephen.Encoder.encode_document(encoder, text,
  skip_punctuation?: true,
  deduplicate?: true
)

Index Options

Standard Index

index = Stephen.Index.new(
  embedding_dim: 128,
  space: :cosine,
  max_tokens: 100_000,
  m: 16,
  ef_construction: 200
)

PLAID Index

plaid = Stephen.Plaid.new(
  embedding_dim: 128,
  num_centroids: 1024
)

Compressed Index

index = Stephen.Index.Compressed.new(
  embedding_dim: 128,
  num_centroids: 1024,
  compression_centroids: 2048,
  residual_bits: 8
)

Search Options

results = Stephen.search(encoder, index, query,
  top_k: 10,
  rerank?: true,
  candidates_per_token: 50
)

PRF Options

Pseudo-relevance feedback expands queries using top-ranked documents:

results = Stephen.search_with_prf(encoder, index, query,
  top_k: 10,
  feedback_docs: 3,
  expansion_tokens: 10,
  expansion_weight: 0.5
)

Tuning tips:

• More feedback_docs adds diversity but may add noise
• Higher expansion_weight emphasizes expansion over original query
• More expansion_tokens broadens matching but may reduce precision

GPU Acceleration

Enable EXLA for GPU acceleration:

# mix.exs
{:exla, "~> 0.9"}

# config/config.exs
config :nx, default_backend: EXLA.Backend

For CPU-only with EXLA optimizations:

config :nx, default_backend: {EXLA.Backend, client: :host}

Batch Processing

For large document sets, process in batches to manage memory:

documents
|> Stream.chunk_every(100)
|> Enum.reduce(index, fn batch, acc ->
  Stephen.index(encoder, acc, batch)
end)

For batch queries:

results = Stephen.Retriever.batch_search(encoder, index, queries, top_k: 10)