Integration helpers specifically designed for DSPEx framework usage.
This module provides utilities that bridge Sinter's validation capabilities with DSPEx's dynamic programming and teleprompter optimization needs.
Summary
Functions
Creates a signature schema for DSPEx programs.
Optimizes a schema based on validation failure patterns.
Prepares a schema for specific LLM provider optimization.
Validates LLM output and enhances errors with debugging context.
Functions
@spec create_signature([tuple()], [tuple()], keyword()) :: Sinter.Schema.t()
Creates a signature schema for DSPEx programs.
A signature defines the input and output structure for a DSPEx program, combining both into a single schema suitable for validation and optimization.
Parameters
input_fields- List of input field specificationsoutput_fields- List of output field specificationsopts- Schema options
Returns
- A schema representing the complete program signature
Examples
iex> signature = Sinter.DSPEx.create_signature(
...> [
...> {:query, :string, [required: true]},
...> {:context, {:array, :string}, [optional: true]}
...> ],
...> [
...> {:answer, :string, [required: true]},
...> {:confidence, :float, [required: true, gteq: 0.0, lteq: 1.0]}
...> ],
...> title: "QA Program Signature"
...> )
iex> fields = Sinter.Schema.fields(signature)
iex> Map.keys(fields)
[:query, :context, :answer, :confidence]
@spec optimize_schema_from_failures(Sinter.Schema.t(), [map()], keyword()) :: {:ok, Sinter.Schema.t(), [String.t()]} | {:error, String.t()}
Optimizes a schema based on validation failure patterns.
Analyzes common validation failures to suggest schema improvements, useful for teleprompter optimization loops.
Parameters
original_schema- The current schemafailure_examples- List of data that failed validationopts- Optimization options
Options
:relaxation_strategy- How to relax constraints (:conservative,:moderate,:aggressive):add_missing_fields- Whether to add commonly missing fields as optional
Returns
{:ok, optimized_schema, suggestions}or{:error, reason}
@spec prepare_for_llm(Sinter.Schema.t(), atom(), keyword()) :: map()
Prepares a schema for specific LLM provider optimization.
Generates both the JSON Schema and metadata needed for optimal structured output with different LLM providers.
Parameters
schema- The Sinter schemaprovider- The target LLM provideropts- Provider-specific options
Returns
- Map with JSON Schema and provider-specific metadata
Examples
iex> schema = Sinter.Schema.define([{:name, :string, [required: true]}])
iex> result = Sinter.DSPEx.prepare_for_llm(schema, :openai)
iex> result.json_schema["additionalProperties"]
false
iex> result.provider
:openai
@spec validate_llm_output(Sinter.Schema.t(), map(), String.t(), keyword()) :: {:ok, map()} | {:error, [Sinter.Error.t()]}
Validates LLM output and enhances errors with debugging context.
This is the primary validation function for DSPEx programs, combining standard validation with LLM-specific error enhancement.
Parameters
schema- The validation schemallm_output- Raw output from LLMoriginal_prompt- The prompt sent to the LLMopts- Validation options
Returns
{:ok, validated_data}or{:error, enhanced_errors}
Examples
iex> schema = Sinter.Schema.define([{:name, :string, [required: true]}])
iex> llm_output = %{"age" => 30} # missing required name
iex> prompt = "Generate a user profile"
iex> {:error, errors} = Sinter.DSPEx.validate_llm_output(schema, llm_output, prompt)
iex> List.first(errors).context.prompt
"Generate a user profile"