View Source Handoff (Handoff v0.1.0)
Handoff is a library for building and executing Directed Acyclic Graphs (DAGs) of functions.
It provides tools for defining computation graphs, managing resources, and executing the graphs in a distributed environment.
Link to this section Summary
Functions
Discovers and registers nodes in the cluster with their capabilities.
Executes all functions in a DAG, respecting dependencies.
Executes all functions in a DAG across multiple nodes, respecting dependencies.
Executes all functions in a DAG strictly on the local node, bypassing resource allocation
and ensuring all functions have node set to Node.self() and cost set to nil.
Retrieves a value from the local store only for a specific DAG.
Retrieves a result for a specific DAG, automatically fetching it from its origin node if necessary.
Retrieves a value for a specific DAG, with automatic remote fetching if needed.
Looks up where a data item (argument or result) is stored for a specific DAG.
Creates a new DAG instance.
Registers the location of a data item (argument or result) for a specific DAG.
Registers the local node with its capabilities for distributed execution.
Registers a node with its resource capabilities.
Checks if the specified node has the required resources available.
Starts the Handoff supervision tree.
Stores a function result locally on the origin node for a specific DAG and registers its location. The result is stored only on the node where it was produced, not broadcast.
Directly stores a value in the local store for a specific DAG.
Link to this section Functions
Discovers and registers nodes in the cluster with their capabilities.
returns
Returns
{:ok, discovered}with a map of node names to their capabilities
Executes all functions in a DAG, respecting dependencies.
parameters
Parameters
- dag: The DAG to execute
- opts: Optional execution settings
- :allocation_strategy - Strategy for allocating functions to nodes (:first_available or :load_balanced, defaults to :first_available)
returns
Returns
{:ok, %{dag_id: dag_id, results: results_map}}with the DAG ID and a map of function IDs to results on success{:error, reason}on failure
Executes all functions in a DAG across multiple nodes, respecting dependencies.
parameters
Parameters
- dag: The DAG to execute
- opts: Optional execution settings
- :allocation_strategy - Strategy for allocating functions to nodes (:first_available or :load_balanced, defaults to :first_available)
- :max_retries - Maximum number of times to retry failed functions (default: 3)
returns
Returns
{:ok, %{dag_id: dag_id, results: results_map}}with the DAG ID and a map of function IDs to results on success{:error, reason}on failure
Executes all functions in a DAG strictly on the local node, bypassing resource allocation
and ensuring all functions have node set to Node.self() and cost set to nil.
This is useful for ensuring local execution regardless of global configuration or function definitions.
parameters
Parameters
- dag: The DAG to execute
- opts: Optional execution settings (passed to
Handoff.DistributedExecutor)
returns
Returns
- Same as
Handoff.execute/2.
Retrieves a value from the local store only for a specific DAG.
parameters
Parameters
- dag_id: The ID of the DAG
- id: The ID of the value to retrieve
returns
Returns
{:ok, value}if found locally{:error, :not_found}if not found
Retrieves a result for a specific DAG, automatically fetching it from its origin node if necessary.
parameters
Parameters
- dag_id: The ID of the DAG
- id: The ID of the result/argument to retrieve
- timeout: Maximum time to wait in milliseconds, defaults to 5000
returns
Returns
{:ok, result}on success{:error, :timeout}if the result is not available within the timeout
Retrieves a value for a specific DAG, with automatic remote fetching if needed.
parameters
Parameters
dag_id: The ID of the DAGid: The ID of the value to retrievefrom_node: Optional specific node to fetch from
returns
Returns
{:ok, value}if found or successfully fetched{:error, reason}if retrieval failed
Looks up where a data item (argument or result) is stored for a specific DAG.
parameters
Parameters
dag_id: The ID of the DAGdata_id: The ID of the data to look up
returns
Returns
{:ok, node_id}if the data location is found{:error, :not_found}if the data location is not registered
Creates a new DAG instance.
Registers the location of a data item (argument or result) for a specific DAG.
parameters
Parameters
- dag_id: The ID of the DAG
- data_id: The ID of the data
- node_id: The node where the data is stored
Registers the local node with its capabilities for distributed execution.
parameters
Parameters
- caps: Map of capabilities provided by this node
example
Example
Handoff.register_local_node(%{cpu: 8, memory: 16000})Registers a node with its resource capabilities.
parameters
Parameters
- node: The node to register
- caps: Map of capabilities/resources the node provides
example
Example
Handoff.register_node(Node.self(), %{cpu: 4, memory: 8000})Checks if the specified node has the required resources available.
parameters
Parameters
- node: The node to check
- req: Map of resource requirements to check
returns
Returns
- true if resources are available
- false otherwise
example
Example
Handoff.resources_available?(Node.self(), %{cpu: 2, memory: 4000})Starts the Handoff supervision tree.
Must be called before executing any DAGs.
store_result(dag_id, function_id, result, origin_node \\ Node.self())
View SourceStores a function result locally on the origin node for a specific DAG and registers its location. The result is stored only on the node where it was produced, not broadcast.
parameters
Parameters
- dag_id: The ID of the DAG
- function_id: The ID of the function
- result: The result to store
- origin_node: The node where the result was produced (defaults to current node)
Directly stores a value in the local store for a specific DAG.
parameters
Parameters
- dag_id: The ID of the DAG
- id: The ID of the value
- value: The value to store