Sinter.DSPEx (Sinter v0.0.1)

View Source

Integration helpers specifically designed for DSPEx framework usage.

This module provides utilities that bridge Sinter's validation capabilities with DSPEx's dynamic programming and teleprompter optimization needs.

Summary

Functions

Creates a signature schema for DSPEx programs.

Optimizes a schema based on validation failure patterns.

Prepares a schema for specific LLM provider optimization.

Validates LLM output and enhances errors with debugging context.

Functions

create_signature(input_fields, output_fields, opts \\ [])

@spec create_signature([tuple()], [tuple()], keyword()) :: Sinter.Schema.t()

Creates a signature schema for DSPEx programs.

A signature defines the input and output structure for a DSPEx program, combining both into a single schema suitable for validation and optimization.

Parameters

  • input_fields - List of input field specifications
  • output_fields - List of output field specifications
  • opts - Schema options

Returns

  • A schema representing the complete program signature

Examples

iex> signature = Sinter.DSPEx.create_signature(
...>   [
...>     {:query, :string, [required: true]},
...>     {:context, {:array, :string}, [optional: true]}
...>   ],
...>   [
...>     {:answer, :string, [required: true]},
...>     {:confidence, :float, [required: true, gteq: 0.0, lteq: 1.0]}
...>   ],
...>   title: "QA Program Signature"
...> )
iex> fields = Sinter.Schema.fields(signature)
iex> Map.keys(fields)
[:query, :context, :answer, :confidence]

optimize_schema_from_failures(original_schema, failure_examples, opts \\ [])

@spec optimize_schema_from_failures(Sinter.Schema.t(), [map()], keyword()) ::
  {:ok, Sinter.Schema.t(), [String.t()]} | {:error, String.t()}

Optimizes a schema based on validation failure patterns.

Analyzes common validation failures to suggest schema improvements, useful for teleprompter optimization loops.

Parameters

  • original_schema - The current schema
  • failure_examples - List of data that failed validation
  • opts - Optimization options

Options

  • :relaxation_strategy - How to relax constraints (:conservative, :moderate, :aggressive)
  • :add_missing_fields - Whether to add commonly missing fields as optional

Returns

  • {:ok, optimized_schema, suggestions} or {:error, reason}

prepare_for_llm(schema, provider, opts \\ [])

@spec prepare_for_llm(Sinter.Schema.t(), atom(), keyword()) :: map()

Prepares a schema for specific LLM provider optimization.

Generates both the JSON Schema and metadata needed for optimal structured output with different LLM providers.

Parameters

  • schema - The Sinter schema
  • provider - The target LLM provider
  • opts - Provider-specific options

Returns

  • Map with JSON Schema and provider-specific metadata

Examples

iex> schema = Sinter.Schema.define([{:name, :string, [required: true]}])
iex> result = Sinter.DSPEx.prepare_for_llm(schema, :openai)
iex> result.json_schema["additionalProperties"]
false
iex> result.provider
:openai

validate_llm_output(schema, llm_output, original_prompt, opts \\ [])

@spec validate_llm_output(Sinter.Schema.t(), map(), String.t(), keyword()) ::
  {:ok, map()} | {:error, [Sinter.Error.t()]}

Validates LLM output and enhances errors with debugging context.

This is the primary validation function for DSPEx programs, combining standard validation with LLM-specific error enhancement.

Parameters

  • schema - The validation schema
  • llm_output - Raw output from LLM
  • original_prompt - The prompt sent to the LLM
  • opts - Validation options

Returns

  • {:ok, validated_data} or {:error, enhanced_errors}

Examples

iex> schema = Sinter.Schema.define([{:name, :string, [required: true]}])
iex> llm_output = %{"age" => 30}  # missing required name
iex> prompt = "Generate a user profile"
iex> {:error, errors} = Sinter.DSPEx.validate_llm_output(schema, llm_output, prompt)
iex> List.first(errors).context.prompt
"Generate a user profile"