altworx_book v0.2.0

AltworxBook.Normalizer.Pipeline (altworx_book v0.2.0)

Normalizer pipeline helper functions.

Link to this section Summary

Types

pipeline_evaluation_result()
pipeline_step()
pipeline_step_result()

Functions

run(messages, steps)

Execute pipeline definition on list of messages.

run_step(step_input, step_fn)

Evaluate pipeline step on given message.

validate_output_message(msg)

Validate message result produced by AltworxBook.Normalizer.Pipeline.run_step/2

visualize_results(result)

Return kino presentation of pipeline evaluation results.

Link to this section Types

Link to this type

pipeline_evaluation_result()

@type pipeline_evaluation_result() ::
  {:new, Toolbox.Message.t()}
  | {:ignored, :ignored, Toolbox.Message.t()}
  | {:error, {:error, term()}, Toolbox.Message.t()}
Link to this type

pipeline_step()

@type pipeline_step() :: (... -> any())
Link to this type

pipeline_step_result()

@type pipeline_step_result() ::
  {:new, Toolbox.Message.t(), Toolbox.Message.t()}
  | {:ignored, :ignored, Toolbox.Message.t()}
  | {:error, term(), Toolbox.Message.t()}

Link to this section Functions

Link to this function

run(messages, steps)

@spec run([AltworxBook.Kafka.RawTopic.message()], [pipeline_step()]) :: [
  pipeline_step_result()
]

Execute pipeline definition on list of messages.

Link to this function

run_step(step_input, step_fn)

@spec run_step(
  AltworxBook.Kafka.RawTopic.message()
  | {:error, term()}
  | :ignored
  | Toolbox.Message.t(),
  pipeline_step()
) :: {:error, term()} | :ignored | Toolbox.Message.t()

Evaluate pipeline step on given message.

Link to this function

validate_output_message(msg)

@spec validate_output_message(Toolbox.Message.t() | term()) :: :ok | {:error, term()}

Validate message result produced by AltworxBook.Normalizer.Pipeline.run_step/2

Link to this function

visualize_results(result)

@spec visualize_results([pipeline_step_result()]) :: Kino.Layout.t()

Return kino presentation of pipeline evaluation results.