Sinter.DSPEx (Sinter v0.0.1)
View SourceIntegration helpers specifically designed for DSPEx framework usage.
This module provides utilities that bridge Sinter's validation capabilities with DSPEx's dynamic programming and teleprompter optimization needs.
Summary
Functions
Creates a signature schema for DSPEx programs.
Optimizes a schema based on validation failure patterns.
Prepares a schema for specific LLM provider optimization.
Validates LLM output and enhances errors with debugging context.
Functions
@spec create_signature([tuple()], [tuple()], keyword()) :: Sinter.Schema.t()
Creates a signature schema for DSPEx programs.
A signature defines the input and output structure for a DSPEx program, combining both into a single schema suitable for validation and optimization.
Parameters
input_fields
- List of input field specificationsoutput_fields
- List of output field specificationsopts
- Schema options
Returns
- A schema representing the complete program signature
Examples
iex> signature = Sinter.DSPEx.create_signature(
...> [
...> {:query, :string, [required: true]},
...> {:context, {:array, :string}, [optional: true]}
...> ],
...> [
...> {:answer, :string, [required: true]},
...> {:confidence, :float, [required: true, gteq: 0.0, lteq: 1.0]}
...> ],
...> title: "QA Program Signature"
...> )
iex> fields = Sinter.Schema.fields(signature)
iex> Map.keys(fields)
[:query, :context, :answer, :confidence]
@spec optimize_schema_from_failures(Sinter.Schema.t(), [map()], keyword()) :: {:ok, Sinter.Schema.t(), [String.t()]} | {:error, String.t()}
Optimizes a schema based on validation failure patterns.
Analyzes common validation failures to suggest schema improvements, useful for teleprompter optimization loops.
Parameters
original_schema
- The current schemafailure_examples
- List of data that failed validationopts
- Optimization options
Options
:relaxation_strategy
- How to relax constraints (:conservative
,:moderate
,:aggressive
):add_missing_fields
- Whether to add commonly missing fields as optional
Returns
{:ok, optimized_schema, suggestions}
or{:error, reason}
@spec prepare_for_llm(Sinter.Schema.t(), atom(), keyword()) :: map()
Prepares a schema for specific LLM provider optimization.
Generates both the JSON Schema and metadata needed for optimal structured output with different LLM providers.
Parameters
schema
- The Sinter schemaprovider
- The target LLM provideropts
- Provider-specific options
Returns
- Map with JSON Schema and provider-specific metadata
Examples
iex> schema = Sinter.Schema.define([{:name, :string, [required: true]}])
iex> result = Sinter.DSPEx.prepare_for_llm(schema, :openai)
iex> result.json_schema["additionalProperties"]
false
iex> result.provider
:openai
@spec validate_llm_output(Sinter.Schema.t(), map(), String.t(), keyword()) :: {:ok, map()} | {:error, [Sinter.Error.t()]}
Validates LLM output and enhances errors with debugging context.
This is the primary validation function for DSPEx programs, combining standard validation with LLM-specific error enhancement.
Parameters
schema
- The validation schemallm_output
- Raw output from LLMoriginal_prompt
- The prompt sent to the LLMopts
- Validation options
Returns
{:ok, validated_data}
or{:error, enhanced_errors}
Examples
iex> schema = Sinter.Schema.define([{:name, :string, [required: true]}])
iex> llm_output = %{"age" => 30} # missing required name
iex> prompt = "Generate a user profile"
iex> {:error, errors} = Sinter.DSPEx.validate_llm_output(schema, llm_output, prompt)
iex> List.first(errors).context.prompt
"Generate a user profile"