Orchid.Recipe (Orchid v0.6.1)

Defines a processing workflow (Recipe).

A Recipe is essentially a collection of Orchid.Steps that describes a Directed Acyclic Graph (DAG) of data processing.

It holds the definition of what needs to be done, but not the execution state.

Example

steps = [
  {MySteps.Download, :url, :raw_html},
  {MySteps.Parse, :raw_html, :data}
]

recipe = Orchid.Recipe.new(steps, name: :scraper_flow)

Options

:name - The name of the recipe (atom).

Summary

Functions

assign_options(recipe, selector, new_opts)

Injects options into steps globally (supports deep traversal).

new(steps, opts \\ [])

Creates a new Recipe.

validate_steps(steps, initial_keys)

Statically validates the steps within a recipe.

walk(steps, func, mode \\ :step)

Performs a deep traversal on a list of steps.

Types

t()

@type t() :: %Orchid.Recipe{
  name: atom() | nil,
  opts: keyword(),
  steps: [Orchid.Step.t()]
}

Functions

assign_options(recipe, selector, new_opts)

@spec assign_options(
  t(),
  Orchid.Step.implementation() | (Orchid.Step.t() -> boolean()) | :all,
  keyword() | %{} | (Orchid.Step.t() -> Orchid.Step.t())
) :: t()

Injects options into steps globally (supports deep traversal).

This function allows you to modify the configuration of specific steps within a recipe, including steps inside nested recipes.

Selectors

The selector determines which steps will receive the new_opts:

:all - Matches every step.
module (atom) - Matches steps implemented by this specific module.
function (arity 2) - Matches steps implemented by this specific function reference.
predicate function (fn step -> boolean()) - A custom function that receives the step and returns true if it should be modified.

new(steps, opts \\ [])

@spec new(
  [Orchid.Step.t()],
  keyword()
) :: t()

Creates a new Recipe.

Arguments

steps - A list of Orchid.Step definitions.
opts - Keyword options (e.g., name: :my_recipe).

validate_steps(steps, initial_keys)

@spec validate_steps([Orchid.Step.t()], [Orchid.Step.io_key()]) ::
  :ok | {:error, term()}

Statically validates the steps within a recipe.

It performs the following checks:

Option Validation: Calls Step.validate_options/1 for each step.
Missing Inputs: Checks if all steps have their required input keys satisfied (either by initial params or previous steps).
Cyclic Dependencies: Checks if the graph contains any cycles.

walk(steps, func, mode \\ :step)

@spec walk(
  [Orchid.Step.t()] | [{Orchid.Step.t(), non_neg_integer()}],
  (Orchid.Step.t() -> Orchid.Step.t()) | (t() -> t()),
  :step | :inner_recipe
) :: [Orchid.Step.t()]

Performs a deep traversal on a list of steps.

This function applies func to every step in the tree. If a step is a NestedStep or its contains an inner recipe(with NestedStep.nested_check/1 returned true), it recursively traverses the inner steps as well.

It supports both standard step lists and indexed step lists (used by Orchid.Scheduler).

Modes

:step (Default) - The func receives and modifies the Step definition.
:inner_recipe - The func receives and modifies the Inner Recipe struct of a nested step.