Runic.Workflow.Rehydration (Runic v0.1.0-alpha.7)

Copy Markdown View Source

Lineage classification and hybrid rehydration for checkpointed workflows.

Classifies facts into hot (needed for forward execution) and cold (historical) sets, enabling memory-efficient recovery by keeping only hot fact values in memory while replacing cold facts with lightweight FactRef structs.

Hot Fact Categories

  1. Pending runnable inputs — Facts on :runnable or :matchable edges, waiting to be consumed by downstream nodes.

  2. Active frontier — Latest-generation facts in each causal lineage (facts that are not parents of any other fact in the graph).

  3. Meta-ref targets — Facts produced by nodes referenced via :meta_ref edges, needed for runtime meta-context resolution. Classification is kind-aware: no values needed for :fact_count / :step_ran? kinds.

  4. Pending join inputs — Facts on :joined edges waiting for join completion.

Usage

alias Runic.Workflow.Rehydration

# Classify facts in a rebuilt workflow
%{hot: hot, cold: cold} = Rehydration.classify(workflow)

# Dehydrate cold facts to FactRefs (frees memory)
workflow = Rehydration.dehydrate(workflow, cold)

# Or use the combined rehydrate/3 for the full flow
{workflow, resolver} = Rehydration.rehydrate(workflow, {StoreMod, store_state})

Summary

Types

classification()

Functions

classify(workflow, opts \\ [])

Classifies all fact vertices in the workflow into hot and cold sets.

dehydrate(workflow, cold_hashes)

Replaces cold Fact vertices with lightweight FactRef structs in the graph.

rehydrate(workflow, store, opts \\ [])

Classifies, dehydrates, and prepares a resolver for a rebuilt workflow.

rehydrate_fused(workflow, store, opts \\ [])

Single-pass classify+dehydrate. Avoids intermediate hot/cold MapSet allocations.

resolve_hot(workflow, hot_hashes, resolver)

Resolves hot FactRef vertices back to full Fact structs.

should_rehydrate?(workflow, opts \\ [])

Heuristic check: returns true if hybrid rehydration is likely to produce meaningful memory savings for this workflow.

Types

classification()

@type classification() :: %{hot: MapSet.t(), cold: MapSet.t()}

Functions

classify(workflow, opts \\ [])

@spec classify(
  Runic.Workflow.t(),
  keyword()
) :: classification()

Classifies all fact vertices in the workflow into hot and cold sets.

Hot facts are needed for forward execution. Cold facts are historical and can be safely replaced with FactRef structs.

Returns %{hot: MapSet.t(hash), cold: MapSet.t(hash)}.

dehydrate(workflow, cold_hashes)

@spec dehydrate(Runic.Workflow.t(), MapSet.t()) :: Runic.Workflow.t()

Replaces cold Fact vertices with lightweight FactRef structs in the graph.

Preserves graph topology — edges reference vertex ids (hashes), which are identical between a Fact and its corresponding FactRef. Only the vertex value in the vertices map is swapped; no edges are modified.

rehydrate(workflow, store, opts \\ [])

@spec rehydrate(Runic.Workflow.t(), {module(), term()}, keyword()) ::
  {Runic.Workflow.t(), Runic.Workflow.FactResolver.t()}

Classifies, dehydrates, and prepares a resolver for a rebuilt workflow.

Combines classify/2 and dehydrate/2 into a single call, returning the dehydrated workflow paired with a FactResolver that can resolve any FactRef on demand from the backing store.

Example

workflow = Workflow.from_events(events)
{workflow, resolver} = Rehydration.rehydrate(workflow, {ETS, store_state})

rehydrate_fused(workflow, store, opts \\ [])

@spec rehydrate_fused(Runic.Workflow.t(), {module(), term()}, keyword()) ::
  {Runic.Workflow.t(), Runic.Workflow.FactResolver.t()}

Single-pass classify+dehydrate. Avoids intermediate hot/cold MapSet allocations.

Pre-computes edge-based hot criteria, then performs two mini-passes over vertices:

  1. Collect parent hashes (needed for frontier detection)
  2. Dehydrate cold facts inline based on hot criteria

Returns the dehydrated workflow paired with a FactResolver.

resolve_hot(workflow, hot_hashes, resolver)

@spec resolve_hot(Runic.Workflow.t(), MapSet.t(), Runic.Workflow.FactResolver.t()) ::
  {Runic.Workflow.t(), Runic.Workflow.FactResolver.t()}

Resolves hot FactRef vertices back to full Fact structs.

Used after lean replay to load only the values needed for forward execution. Cold FactRefs remain as lightweight references.

should_rehydrate?(workflow, opts \\ [])

@spec should_rehydrate?(
  Runic.Workflow.t(),
  keyword()
) :: boolean()

Heuristic check: returns true if hybrid rehydration is likely to produce meaningful memory savings for this workflow.

Samples fact values and checks total fact count against thresholds derived from benchmark data.