Crucible.Pipeline.Runner (CrucibleFramework v0.5.2)

View Source

Executes experiment pipelines stage-by-stage.

Location and Ownership

This is the authoritative pipeline runner for the Crucible ecosystem. It lives in crucible_framework and is the only component that executes experiment pipelines. crucible_ir defines specs only; it does not execute.

Public Entrypoint

Users should call CrucibleFramework.run/2 rather than this module directly:

{:ok, ctx} = CrucibleFramework.run(experiment)

Pipeline Execution

The runner:

  1. Initializes a %Crucible.Context{} from the experiment
  2. Optionally persists run state to the database
  3. Executes each %CrucibleIR.StageDef{} in sequence
  4. Resolves stage modules via Crucible.Registry or explicit :module field
  5. Optionally validates stage options against describe/1 schema
  6. Calls stage_module.run(context, opts) for each stage
  7. Marks stages complete and emits trace events
  8. Finalizes the run with success or failure status

Stage Resolution

Stages are resolved in order:

  1. If StageDef.module is set, use that module directly
  2. Otherwise, look up StageDef.name in Crucible.Registry

Options Validation

The runner supports opt-in validation of stage options against the schema returned by each stage's describe/1 callback:

CrucibleFramework.run(experiment, validate_options: :error)
  • :off (default) - No validation
  • :warn - Log warnings for invalid options but continue execution
  • :error - Fail immediately on invalid options

Trace Integration

When :enable_trace is passed, the runner emits stage lifecycle events via Crucible.TraceIntegration for observability and debugging.

Summary

Functions

Runs an experiment, optionally persisting run state.

Functions

run(experiment, opts \\ [])

@spec run(
  CrucibleIR.Experiment.t(),
  keyword()
) :: {:ok, Crucible.Context.t()} | {:error, term()}

Runs an experiment, optionally persisting run state.

Options

  • :run_id - Custom run ID (defaults to UUID)
  • :persist - Whether to persist run state (default: true)
  • :enable_trace - Enable trace integration (default: false)
  • :assigns - Initial context assigns (default: %{})
  • :validate_options - Options validation mode:
    • :off (default) - No validation
    • :warn - Log warnings but continue
    • :error - Fail on validation errors