Specs

activate(RaftFleet.ZoneId.t()) :: :ok | {:error, :not_inactive}

Activates Node.self().

When :raft_fleet is started as an OTP application, the node is not active; to host consensus group members each node must be explicitly activated. zone is an ID of data center zone which this node belongs to. zone is used to determine which nodes to replicate data: RaftFleet tries to place members of each consensus group across multiple zones for maximum availability.

Node activation by calling this function should be done after the node is fully connected to the other existing nodes. Otherwise there is a possibility that the current node, without noticing other active nodes in the cluster, initiates a new 1-member cluster instead of joining the already initiated cluster.

Note that, before calling this function, it is necessary to load and start OTP applications that are required by consensus group implementations. Specifically, :data_modules and :communication_modules specified by consensus group configurations must be available.

Specs

active_nodes() :: %{required(RaftFleet.ZoneId.t()) => [node()]}

Queries the current nodes which have been activated using activate/1 in the cluster.

This function sends a query to a leader of the RaftFleet.Cluster consensus group, which is managed internally by raft_fleet. The returned value is grouped by zone IDs which have been passed to activate/1. This function exits if no active node exists in the cluster.

Specs

add_consensus_group(atom()) ::
  :ok | {:error, :already_added | :cleanup_ongoing | :no_leader | any()}

Registers a new consensus group identified by name.

This is a simplified version of add_consensus_group/3, where

• n_replica is filled with 3 (fixed value) and
• rv_config is computed using the module given as :rafted_value_config_maker option (see also RaftFleet.Config).

Specs

add_consensus_group(
  atom(),
  pos_integer(),
  RaftedValue.Config.t(),
  pos_integer()
) :: :ok | {:error, :already_added | :cleanup_ongoing | :no_leader | any()}

Registers a new consensus group identified by name.

name is used as the registered name for member processes of the new consensus group. n_replica is the number of replicas (Raft member processes implemented as RaftedValue.Server). If you want all active nodes to host member processes, specify sufficiently large integer as n_replica. For explanation of rv_config see RaftedValue.make_config/2.

If you configure raft_fleet to persist Raft logs & snapshots (see also RaftFleet.Config) and the consensus group with name had been removed by remove_consensus_group/1, then add_consensus_group/3 will restore the state of the consensus group from the snapshot and log files.

The caller is blocked until the newly spawned leader becomes ready. await_timeout specifies how many milliseconds to wait for the initialization.

Specs

command(
  atom(),
  RaftedValue.Data.command_arg(),
  pos_integer(),
  non_neg_integer(),
  pos_integer(),
  module()
) :: {:ok, RaftedValue.Data.command_ret()} | {:error, :no_leader}

Executes a command on the replicated value identified by name.

The target consensus group identified by name must be registered beforehand using add_consensus_group/3. This function automatically resolves the leader process of the consensus group, caches PID of the current leader in local ETS table and send the given command to the leader.

timeout is used in each synchronous messaging. In order to tolerate temporal absences of leaders during Raft leader elections, it retries requests up to retry. Before retrying requests this function sleeps for retry_interval milliseconds. Thus for worst case this function blocks the caller for timeout * (retry + 1) + retry_interval * retry. Note that for complete masking of leader elections retry_interval * retry must be sufficiently longer than the time scale for leader elections (:election_timeout in RaftedValue.Config.t).

call_module can be changed from the default (:gen_statem module) with an alternative module that exports call/3 for optimization of remote message passing.

See also RaftedValue.command/5.

Specs

consensus_groups() :: %{required(atom()) => pos_integer()}

Queries already registered consensus groups.

This function sends a query to a leader of the RaftFleet.Cluster consensus group, which is managed internally by raft_fleet. The returned value is a map whose keys and values are consensus group name and number of replicas of the group. This function exits if no active node exists in the cluster.

Specs

deactivate() :: :ok | {:error, :inactive}

Deactivates Node.self().

Call this function before you remove an ErlangVM from your cluster. Note that calling this function does not immediately remove consensus member processes in this node; these processes will be gradually migrated to other nodes by periodic rebalancing.

Specs

find_consensus_group_with_no_established_leader() ::
  :ok | {group_name :: atom(), [{node(), map()}]}

Inspects members of consensus groups and finds a group (if any) in which no leader exists.

If one found, returns the name of the consensus group and also statuses (as maps) of existing members. If no consensus group is in trouble, returns :ok.

Target consensus groups are:

• RaftFleet.Cluster, which is a special consensus group that manages metadata for other consensus groups
• all registered consensus groups (i.e., the ones returned by RaftFleet.consensus_groups/0)

This function is primarily intended to be used within remote console. Use this function to detect problematic consensus group in your cluster.

Specs

query(
  atom(),
  RaftedValue.Data.query_arg(),
  pos_integer(),
  non_neg_integer(),
  pos_integer(),
  module()
) :: {:ok, RaftedValue.Data.query_ret()} | {:error, :no_leader}

Executes a read-only query on the replicated value identified by name.

See command/6 for explanations of name, timeout, retry, retry_interval and call_module. See also RaftedValue.query/4.

Specs

remove_consensus_group(atom()) :: :ok | {:error, :not_found | :no_leader}

Removes an existing consensus group identified by name.

Removing a consensus group will eventually trigger terminations of all members of the group. The replicated value held by the group will be discarded.

Note that remove_consensus_group/1 does not immediately terminate existing member processes; they will be terminated afterward by background worker process (see also :balancing_interval in RaftFleet.Config). Note also that, if Raft logs and snapshots has been created (see also RaftFleet.Config), remove_consensus_group/1 does not remove these files.

Specs

remove_dead_pids_located_in_dead_node(node()) :: :ok

Removes member pids that reside in the specified dead node from all existing consensus groups.

Target consensus groups are:

• RaftFleet.Cluster, which is a special consensus group that manages metadata for other consensus groups
• all registered consensus groups (i.e., the ones returned by RaftFleet.consensus_groups/0)

If a target consensus group does not have an established leader, then this function tries to remove dead pids (if any) by using RaftedValue.force_remove_member/2.

This function crashes if the RaftFleet.Cluster consensus group does not have a leader. Each of the effects of this function is idempotent; you can freely call this function multiple times in case of failure.

Use this function to resolve issues when e.g. some node suddenly died without cleaning up itself. The caller must be sure that the dead_node has definitely died.

Specs

unreachable_nodes() :: %{required(node()) => unreachable_since}
when unreachable_since: pos_integer()

Gets information about nodes that are active but unreachable from Node.self().

RaftFleet periodically checks reachability to the other active nodes. This function retrieves results of the checks as a map. Each key of the returned map is a currently unreachable node (i.e. if there's no problem %{} is returned), whereas each value is a timestamp (seconds since epoch) at which the node was recognized as unreachable.

Note that returned timestamps are not so accurate about when the node failed, since they are obtained by healthchecks with an interval of :node_purge_reconnect_interval (see also RaftFleet.Config). Note also that, when a node remains unreachable for more than :node_purge_failure_time_window, the node will be automatically purged from the active nodes. After purging the failed node will not be included in return value of this function (as the node is no longer active).

Specs

whereis_leader(atom()) :: nil | pid()

Tries to find the current leader of the consensus group specified by name.

Usually you don't have to use this function as command/5 and query/5 automatically resolves where the leader resides. This function is useful when you want to inspect status of a consensus group by using e.g. RaftedValue.status/1.