A behaviour module for implementing consistently replicated state machines.

Example

To start things off, let's see a code example implementing a simple key value store in ExRaft style:

defmodule KeyValueStore do
  @behaviour ExRaft.StateMachine

  @impl true
  def init(init_state) do
    {:ok, init_state}
  end

  @impl true
  def command?({:put, _, _}, _), do: true
  def command?(_, _), do: false

  @impl true
  def handle_write({:put, key, value}, state) do
    new_state = Map.put(state, key, value)
    reply = {:ok, "I GOT IT BOSS"}
    {reply, new_state}
  end

  @impl true
  def handle_read(key, state) do
    Map.get(state, key)
  end
end

initial_config = [foo: node(), bar: node(), baz: node()]
init_state = %{}
{:ok, _} = ExRaft.start_server(KeyValueStore, init_state, name: :foo, initial_config: initial_config)
{:ok, _} = ExRaft.start_server(KeyValueStore, init_state, name: :bar, initial_config: initial_config)
{:ok, _} = ExRaft.start_server(KeyValueStore, init_state, name: :baz, initial_config: initial_config)

leader = ExRaft.await_leader(:foo)
# leader is now one of [{:foo, node()}, {:bar, node()}, {:baz, node()}]

ExRaft.read(leader, :some_key)
# nil

ExRaft.write(KeyValueStore, {:put, :some_key, :some_value})
# {:ok, "I GOT IT BOSS"}

ExRaft.read(leader, :some_key)
# :some_value

We start a member of our KeyValueStore cluster by calling ExRaft.start_server/3, passing the module with the state machine implementation and its initial state (in this example this is just an empty map).

In the example above we start a 3 server cluster with the names :foo, :bar and :baz. Any of these servers may become the cluster leader at startup or when the old leader fails, that is why we first wait for a leader to be elected by calling ExRaft.await_leader/2.

After a leader was elected we can write to- and read from our replicated key value store by calling ExRaft.write/3 and ExRaft.read/3 respectively.

Once the ExRaft.write/3 call returns with our state machine reply ({:ok, "I GOT IT BOSS"}) we know that the majority of our cluster (2 servers in this example) knows that :some_key is in fact :some_value. This means that if the leader crashes or a network split happens between the leader and the rest of the cluster, the remaining 2 servers will still be able to elect a new leader and this new leader will still know of :some_key being :some_value.

In fact if we were to issue an ExRaft.read_dirty/3 call on the followers after writing :some_key to the state at least one, if not both of them would reply with :some_value.

Improvements

In the above example we interacted with the key value store using the ExRaft module. This is not ideal since we don't want our users to necessarily know how to use ExRaft.

Also we started the servers by calling ExRaft.start_server/3 directly. It would be better if we started them as part of a supervision tree.

So let's fix these issues:

defmodule KeyValueStore do
  use ExRaft.StateMachine

  @init_state %{}

  def start_link(opts),
    do: ExRaft.start_server(__MODULE__, @init_state, opts)

  def put(server, key, value, timeout \\ nil),
    do: ExRaft.write(server, {:put, key, value}, timeout)

  def get(server, key, timeout \\ nil),
    do: ExRaft.read(server, key, timeout)

  @impl true
  def init(init_state) do
    {:ok, init_state}
  end

  @impl true
  def command?({:put, _, _}, _), do: true
  def command?(_, _), do: false

  @impl true
  def handle_write({:put, key, value}, state) do
    new_state = Map.put(state, key, value)
    reply = {:ok, "I GOT IT BOSS"}
    {reply, new_state}
  end

  @impl true
  def handle_read(key, state) do
    Map.get(state, key)
  end
end

Now we can simply call KeyValueStore.put/3 and KeyValueStore.get/3 to write to- and read from our replicated key value store.

Supervision

When we invoke use ExRaft.StateMachine, two things happen:

It defines that the current module implements the ExRaft.StateMachine behaviour (@behaviour ExRaft.StateMachine).

It also defines an overridable child_spec/1 function, that allows us to start the KeyValueStore directly under a supervisor.

children = [
  {KeyValueStore, name: :foo, initial_config: [...]}
]

Supervisor.start_link(children, strategy: :one_for_one)

This will invoke KeyValueStore.start_link/1, lucky for us we already defined this function in the improved implementation.

Note

Be aware that all callbacks (except for init/1 and terminate/2) block the server until they return, so it is recommended to try and keep any heavy lifting outside of them or adjusting the :min_election_timeout and :max_election_timeout options of the server (see ExRaft.start_server/3).

Also note that setting a high election timeout range may cause the cluster to stay without leader and thus become unresponsive for a longer period of time.