Basic Concepts

A Basic Graph

To illustrate a few concepts (graph, dependencies – including conditional dependencies, computation functions, persistence), here is a slightly more complex example.

Keep in mind that these concepts apply to both simple, two-step linear processes and complex, multi-step conditional processes. Journey makes it easy to implement durable flows of a wide range of size and complexity.

This graph adds two numbers when they become available, and conditionally sets the "too large" flag.

iex> import Journey.Node
iex> # Defining a graph, with two input nodes and two downstream computations.
iex> graph = Journey.new_graph(
...>   "demo graph – doctest",
...>   "v1",
...>   [
...>     input(:x),
...>     input(:y),
...>     # :sum is unblocked when :x and :y are provided.
...>     compute(:sum, [:x, :y], fn %{x: x, y: y} -> {:ok, x + y} end),
...>     # :large_value_alert is unblocked when :sum is provided and is greater than 40.
...>     compute(
...>         :large_value_alert,
...>         [sum: fn sum_node -> sum_node.node_value > 40 end],
...>         fn %{sum: sum} -> {:ok, "🚨, at #{sum}"} end,
...>         f_on_save: fn _execution_id, _result ->
...>            # (e.g. send a pubsub notification to the LiveView process to update the UI)
...>            :ok
...>         end
...>     )
...>   ]
...> )
iex> # Start an execution of this graph, set input values, read computed values.
iex> execution = Journey.start_execution(graph)
iex> execution = Journey.set(execution, :x, 12)
iex> execution = Journey.set(execution, :y, 2)
iex> {:ok, 14, _rev} = Journey.get(execution, :sum, wait: :any)
iex> Journey.get(execution, :large_value_alert)
{:error, :not_set}
iex> eid = execution.id
iex> # After an outage / redeployment / page reload / long pause, an execution
iex> # can be reloaded and continue, as if nothing happened.
iex> execution = Journey.load(eid)
iex> # An update to :y triggers a re-computation of downstream values.
iex> execution = Journey.set(execution, :y, 37)
iex> {:ok, "🚨, at 49", _rev} = Journey.get(execution, :large_value_alert, wait: :any)
iex> Journey.values(execution) |> redact([:execution_id, :last_updated_at])
%{execution_id: "...", last_updated_at: 1234567890, sum: 49, x: 12, y: 37, large_value_alert: "🚨, at 49"}

The graph can be visualized as a Mermaid graph:

> Journey.Tools.generate_mermaid_graph(graph)
graph TD
  %% Graph
  subgraph Graph["🧩 'demo graph', version v1"]
      execution_id[execution_id]
      last_updated_at[last_updated_at]
      x[x]
      y[y]
      sum["sum<br/>(anonymous fn)"]
      large_value_alert["large_value_alert<br/>(anonymous fn)"]

      x -->  sum
      y -->  sum
      sum -->  large_value_alert
  end

  %% Styling
  classDef inputNode fill:#e1f5fe,stroke:#01579b,stroke-width:2px,color:#000000
  classDef computeNode fill:#f3e5f5,stroke:#4a148c,stroke-width:2px,color:#000000
  classDef scheduleNode fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#000000
  classDef mutateNode fill:#e8f5e8,stroke:#2e7d32,stroke-width:2px,color:#000000

  %% Apply styles to actual nodes
  class y,x,last_updated_at,execution_id inputNode
  class large_value_alert,sum computeNode

A few things to note about this example:

Every input value (:x, :y), or computation result (:sum, :large_value_alert) is persisted as soon as it becomes available,
The functions attached to :sum and :large_value_alert
- are called reliably, with a retry policy,
- will execute on any of the replicas of your application,
- are called proactively – when their upstream dependencies are available.
Executions of this flow can take as long as needed (milliseconds? months?), and will live through system restarts, crashes, redeployments, page reloads, etc.

What Does Journey Provide?

Despite the simplicity of use, here are a few things provided by Journey that are worth noting:

Persistence: Executions are persisted, so if the customer leaves the website, or if the system crashes, their execution can be reloaded and continued from where it left off.
Scaling: Since Journey runs as part of your application, it scales with your application. Your graph's computations (:sum's function in the first example, or &compute_zodiac_sign/1 and &compute_horoscope/1 in the horoscope example below) run on the same nodes where the replicas of your application are running. No additional infrastructure or cloud services are needed.
Reliability: Journey uses database-based supervision of computation tasks: The compute functions are subject to a customizable retry policy, so if :sum's function above or &compute_horoscope/1 below fails because of a temporary glitch (e.g. the LLM service it uses for drafting horoscopes is currently overloaded), it will be retried.
Code Structure: The flow of your application is captured in the Journey graph, and the business logic is captured in the compute functions (:sum's function above, or &compute_zodiac_sign/1 and &compute_horoscope/1 below). This clean separation supports you in structuring the functionality of your application in a clear, easy-to-understand, and easy-to-maintain way.
Conditional flow: Journey allows you to define conditions for when a node is to be unblocked. So if your graph includes a "credit_approval_decision" node, the decision can inform which part of the graph is to be executed next (sending a "congrats!" email and starting the credit card issuance process, or sending a "sad trombone" email).
Graph Visualization: Journey provides tools for visualizing your application's graph, so you can easily see the flow of data and computations in your application, and to share and discuss it with your team.
Scheduling: Your graph can include computations that are scheduled to run at a later time, or on a recurring basis. Daily horoscope emails! A reminder email if they haven't visited the website in a while! A "happy birthday" email!
Removing PII: Journey gives you an easy way to erase sensitive data once it is no longer needed. For example, your Credit Card Application graph can include a step to remove the SSN once the credit score has been computed. For an example, please see
```
mutate(:ssn_redacted, [:credit_score], fn _ -> {:ok, "<redacted>"} end, mutates: :ssn)
```
node in the example credit card application graph, here, which mutates the contents of the :ssn node, replacing its value with "<redacted>", when :credit_score completes.
Tooling and visualization: Journey.Tools provides a set of tools for introspecting and managing executions, and for visualizing your application's graph.

A (slightly) richer example: computing horoscopes

Consider a simple Horoscope application that computes a customer's zodiac sign and horoscope based on their birthday. The application will ask the customer to input their name and birthday, and it then auto-computes their zodiac sign and horoscope.

This application can be thought of as a graph of nodes, where each node represents a piece of customer-provided data or the result of a computation. Add functions for computing the zodiac sign and horoscope, and capture the sequencing of the computations, and you have a graph that captures the flow of data and computations in your application. When a customer visits your application, you can start the execution of the graph, to accept and store customer-provided inputs (name, birthday), and to compute the zodiac sign and horoscope based on these inputs.

Journey provides a way to define such graphs, and to run their executions, to serve your customer flows.

Step-by-Step

Below is a step-by-step example of defining a Journey graph for this Horoscope application.

(These are code snippets, if you want a complete fragment you can paste into iex or livebook, scroll down to the "Putting together" code block.)

This graph captures customer inputs, and defines computeations (together with their functions and prerequisites):

graph = Journey.new_graph(
  "horoscope workflow - module doctest (abbreviated)",
  "v1.0.0",
  [
    input(:first_name),
    input(:birth_day),
    input(:birth_month),
    compute(
      :zodiac_sign,
      [:birth_month, :birth_day],
      &compute_zodiac_sign/1
    ),
    compute(
      :horoscope,
      [:first_name, :zodiac_sign],
      &compute_horoscope/1
    )
  ]
)

When a customer lands on your web page, and starts a new flow, your application will start a new execution of the graph,

execution = Journey.start_execution(graph)

and it will populate the execution with the input values (name, birthday) as the customer provides them:

execution = Journey.set(execution, :first_name, "Mario")
execution = Journey.set(execution, :birth_day, 5)
execution = Journey.set(execution, :birth_month, "May")

Providing these input values will trigger automatic computations of the customer's :zodiac_sign and :horoscope, which can then be read from the execution and rendered on the web page.

{:ok, zodiac_sign, _rev} = Journey.get(execution, :zodiac_sign, wait: :any)
{:ok, horoscope, _rev} = Journey.get(execution, :horoscope, wait: :any)

And that's it!

Putting It All Together

Putting together the components of the horoscope example into a complete, running doctest example:

iex> # 1. Define a graph capturing the data and the logic of the application -
iex> #    the nodes, their dependencies, and their computations:
iex> import Journey.Node
iex> graph = Journey.new_graph(
...>       "horoscope workflow - module doctest (all together now)",
...>       "v1.0.0",
...>       [
...>         input(:first_name),
...>         input(:birth_day),
...>         input(:birth_month),
...>         compute(
...>           :zodiac_sign,
...>           # Depends on user-supplied data:
...>           [:birth_month, :birth_day],
...>           # Computes itself, once the dependencies are satisfied:
...>           fn %{birth_month: _birth_month, birth_day: _birth_day} ->
...>             {:ok, "Taurus"}
...>           end
...>         ),
...>         compute(
...>           :horoscope,
...>           # Computes itself once :first_name and :zodiac_sign are in place:
...>           [:first_name, :zodiac_sign],
...>           fn %{first_name: name, zodiac_sign: zodiac_sign} ->
...>             {:ok, "🍪s await, #{zodiac_sign} #{name}!"}
...>           end
...>         )
...>       ]
...>     )
iex>
iex> # 2. For every customer visiting your website, start a new execution of the graph:
iex> e = Journey.start_execution(graph)
iex>
iex> # 3. Populate the execution's nodes with the data as provided by the visitor:
iex> e = Journey.set(e, :birth_day, 26)
iex>
iex> # As a side note: if the user leaves and comes back later or if everything crashes,
iex> # you can always reload the execution using its id:
iex> e = Journey.load(e.id)
iex>
iex> # Continuing, as if nothing happened:
iex> e = Journey.set(e, :birth_month, "April")
iex>
iex> # 4. Now that we have :birth_month and :birth_day, :zodiac_sign will compute itself:
iex> {:ok, "Taurus", _rev} = Journey.get(e, :zodiac_sign, wait: :any)
iex> Journey.values(e) |> redact([:execution_id, :last_updated_at])
%{birth_day: 26, birth_month: "April", zodiac_sign: "Taurus", execution_id: "...", last_updated_at: 1234567890}
iex>
iex> # 5. Once we get :first_name, the :horoscope node will compute itself:
iex> e = Journey.set(e, :first_name, "Mario")
iex> {:ok, "🍪s await, Taurus Mario!", _rev} = Journey.get(e, :horoscope, wait: :any)
iex>
iex> Journey.values(e) |> redact([:execution_id, :last_updated_at])
%{birth_day: 26, birth_month: "April", first_name: "Mario", horoscope: "🍪s await, Taurus Mario!", zodiac_sign: "Taurus", execution_id: "...", last_updated_at: 1234567890}
iex>
iex> # 6. ... and we can always list executions.
iex> this_execution = Journey.list_executions(graph_name: "horoscope workflow - module doctest (all together now)", order_by_execution_fields: [:inserted_at]) |> Enum.reverse() |> hd
iex> e.id == this_execution.id
true

For a more in-depth example of building a more complex application, see the Credit Card Application example in Journey.Examples.CreditCardApplication.

← Previous Page Overview

Next Page → Modules and Functions