Behaviour module for Kreuzberg document extraction validators.

This module defines the callback interface for implementing custom validators in the Kreuzberg plugin system. Validators are responsible for validating extraction results and ensuring data quality and consistency.

Validators are executed in a pipeline with configurable priorities, allowing fine-grained control over validation order and result handling. Each validator can decide whether it should validate a given result based on custom logic.

Validator Lifecycle

The validator lifecycle consists of four main phases:

1. Initialization - Called once when the validator is registered

   • Use this to set up resources, connect to services, etc.
   • Must return :ok or {:error, reason}

2. Conditional Validation - Before validating, check if validation should run

   • Use should_validate?/1 to conditionally apply validation logic
   • Useful for document-type-specific validators

3. Validation - Perform the actual validation

   • Check result structure and content
   • Return :ok or {:error, reason} with descriptive message

4. Shutdown - Called when the validator is unregistered

   • Use this to clean up resources
   • Must return :ok

Priority System

Validators are sorted by priority (descending) before execution. Higher priority values run first. This allows you to:

• Run fast validators first (fail-fast approach)
• Run validators with dependencies in order
• Control the order of detailed validation passes

Typical priority levels:

• 100+ - Critical validators (must pass)
• 50-100 - High priority validators
• 1-50 - Standard validators
• 0 or negative - Low priority (informational)

Validation Results

All validation functions return one of:

• :ok - Validation passed
• {:error, reason} - Validation failed with human-readable reason

Error messages should be descriptive enough to help developers:

• Specify what was wrong
• Explain why it matters
• Provide hints for fixing the issue

Example Validators

See the examples below for common validator patterns.

Behaviour Callbacks

All modules implementing this behaviour must define:

• name/0 - Return a unique validator identifier
• validate/1 - Perform validation on extraction result
• should_validate?/1 - Decide if validation should run
• priority/0 - Return validation priority (integer)
• initialize/0 - Set up validator resources
• shutdown/0 - Clean up validator resources
• version/0 - Return validator version string

Examples

A minimal validator that checks for empty content:

defmodule MyApp.Validators.NonEmptyValidator do
  @behaviour Kreuzberg.Plugin.Validator

  def name, do: "non_empty_content_validator"

  def validate(result) do
    if String.length(result["content"] || "") > 0 do
      :ok
    else
      {:error, "Extraction result contains empty content"}
    end
  end

  def should_validate?(result) do
    is_map(result) and Map.has_key?(result, "content")
  end

  def priority, do: 100

  def initialize do
    :ok
  end

  def shutdown, do: :ok

  def version, do: "1.0.0"
end

A more complex validator that validates PDF metadata:

defmodule MyApp.Validators.PDFMetadataValidator do
  @behaviour Kreuzberg.Plugin.Validator

  def name, do: "pdf_metadata_validator"

  def validate(result) do
    with {:ok, mime} <- validate_mime_type(result),
         {:ok, metadata} <- validate_metadata_exists(result),
         {:ok, _} <- validate_required_fields(result) do
      :ok
    end
  end

  def should_validate?(result) do
    mime_type = result["mime_type"]
    String.starts_with?(mime_type || "", "application/pdf")
  end

  def priority, do: 75

  def initialize do
    # Could initialize PDF validation library here
    :ok
  end

  def shutdown, do: :ok

  def version, do: "2.1.0"

  # Private helpers would go here
  # (implementation details omitted for brevity)
end

A stateful validator that tracks statistics:

defmodule MyApp.Validators.StatisticsValidator do
  @behaviour Kreuzberg.Plugin.Validator
  use GenServer

  def name, do: "statistics_validator"

  def validate(result) do
    try do
      # Update statistics
      :ok = GenServer.call(__MODULE__, {:track, result})
      :ok
    catch
      :exit, _ ->
        {:error, "Failed to record statistics"}
    end
  end

  def should_validate?(_result), do: true

  def priority, do: 10

  def initialize do
    GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
    :ok
  end

  def shutdown do
    GenServer.stop(__MODULE__)
  end

  def version, do: "1.5.0"

  # GenServer callbacks
  @impl true
  def init(state) do
    {:ok, state}
  end

  @impl true
  def handle_call({:track, result}, _from, state) do
    new_state = update_stats(state, result)
    {:reply, :ok, new_state}
  end

  defp update_stats(state, result) do
    # Track metrics based on result
    state
  end
end

Validator Registration

Validators are registered with the Kreuzberg plugin system:

Kreuzberg.Plugin.register_validator(MyApp.Validators.NonEmptyValidator)

And can be unregistered when no longer needed:

Kreuzberg.Plugin.unregister_validator("non_empty_content_validator")

Validation Pipeline

During result processing, the system:

1. Collects all registered validators
2. Sorts by priority (highest first)
3. For each validator: a. Calls should_validate?/1 to check applicability b. If true, calls validate/1 c. Continues on :ok, may stop on error based on policy
4. Returns combined validation result

Error Handling

When validation fails:

• Single validation error: {:error, "reason"}
• Multiple validation errors: {:error, [{"validator_name", "reason"}, ...]}
• System errors: {:error, "Validator xyz: system error"}

Validators should avoid raising exceptions and instead return error tuples.