Crawly.Pipeline behaviour (Crawly v0.12.0) View Source

A behavior module for implementing a pipeline module

A pipeline is a module which takes a given item, and executes a run callback on a given item.

A state argument is used to share common information accros multiple items.

An opts argument is used to pass configuration to the pipeline through tuple-based declarations.

Link to this section Summary

run(item :: map(), state :: map()) ::
  {new_item :: map(), new_state :: map()} | {false, new_state :: map()}

(optional)

run(item :: map(), state :: map(), args :: [any()]) ::
  {new_item :: map(), new_state :: map()} | {false, new_state :: map()}