Gust.DSL (gust v0.1.25)

The Gust DSL is how you turn a module into a DAG.
When you add use Gust.DSL to a module in the dags/ folder, Gust automatically detects it and creates a DAG based on the file name.

You can configure a schedule, define callbacks, and in the dev environment the code is automatically reloaded when files change.

After enabling the DSL, use task definitions to declare the steps that should be executed.

Example

defmodule HelloWorld do
  # `schedule` and `on_finished_callback` are optional.
  # Note: if you change `schedule`, restart the server to update the cron job.
  use Gust.DSL, schedule: "* * * * *", on_finished_callback: :notify_something

  # Gust logs are stored and displayed through GustWeb via Logger.
  require Logger

  # Gust.Flows is used to query Dag, Run, and Task.
  alias Gust.Flows

  def notify_something(status, run) do
    dag = Flows.get_dag!(run.dag_id)
    message = "DAG: #{dag.name}; completed with status: #{status}"
    Logger.info(message)
  end

  task :first_task, downstream: [:second_task], store_result: true do
    greetings = "Hi from first_task"
    Logger.info(greetings)

    # You can get secrets created on the Web UI
    secret = Flows.get_secret_by_name("SUPER_SECRET")
    Logger.warning("I know your secret: #{secret.value}")

    # The return value must be a map when `store_result` is true.
    %{result: greetings}
  end

  task :second_task, ctx: %{run_id: run_id} do

    # Getting "first_task"'s result
    task = Flows.get_task_by_name_run("first_task", run_id)

    Logger.info(task.result)
  end
end

Parameters

schedule - A valid cron expression string.
on_finished_callback - The name of the function to be called.

Summary

Functions

task(name, list)

Defines a task in the DAG without options or explicit context matching.

task(name, opts_and_ctx, list)

Defines a task in the DAG.

Functions

task(name, list)

(macro)

Defines a task in the DAG without options or explicit context matching.

Parameters

name - The name of the task (atom).
block - The code block to execute for the task.

Example

task :simple_task do
  IO.puts "Hello"
end

task(name, opts_and_ctx, list)

(macro)

Defines a task in the DAG.

Parameters

name — The name of the task (atom).
opts_and_ctx — A keyword list of options and an optional context pattern.
block — The code block executed when the task runs.

Task Options

:downstream — A list of task names (atoms) to run after this task completes.
:store_result — When true, the task's return value will be persisted.
- Note: If enabled, the return value must be a map.
:ctx — A pattern that will be matched against the context passed to the task.
- Defaults to: %{run_id: run_id}.

Example

task :my_task, ctx: %{run_id: run_id} do
  IO.inspect(run_id)
end

task :first, downstream: [:second] do
  :ok
end

task :persist_result, store_result: true do
  %{result: :ok}
end

When using store_result: true, the return value must be a map so it can be merged into the overall DAG results.