Synaptic.Experiments (synaptic v0.2.7)

Experimental API for running workflows against multiple inputs and scorers.

This module intentionally provides a minimal, in-memory harness inspired by Mastra's runExperiment function. It reuses the existing scorer abstraction and the public Synaptic.start/3 + Synaptic.inspect/1 APIs without adding new runtime behaviour to Synaptic.Runner.

The API is kept small on purpose to allow iteration as real-world use cases emerge.

Summary

Types

data_item()

Data item used when running experiments.

summary()

High-level summary of an experiment run.

Functions

run_experiment(workflow_module, data_items, opts \\ [])

Sketch of an experiment runner that evaluates a workflow against many inputs.

Types

data_item()

@type data_item() :: %{:input => map(), optional(:metadata) => map()}

Data item used when running experiments.

The :input map is passed as the initial workflow context. Arbitrary metadata can be attached via :metadata and will be available to scorers through their own configuration or via Telemetry subscribers.

summary()

@type summary() :: %{total_items: non_neg_integer()}

High-level summary of an experiment run.

Functions

run_experiment(workflow_module, data_items, opts \\ [])

@spec run_experiment(module(), [data_item()], keyword()) :: %{summary: summary()}

Sketch of an experiment runner that evaluates a workflow against many inputs.

This function is intentionally conservative: it starts a real Synaptic workflow for each input using Synaptic.start/3 and waits for completion using Synaptic.inspect/1. Scorer execution is still driven by Synaptic.Runner just like in normal production runs.

Returns an opaque result map for now, primarily for future extension. The main integration point for host applications is still Telemetry (e.g. [:synaptic, :scorer] and workflow-level events).