raft_kv v0.2.5 RaftKV.SplitMergePolicy View Source

An Elixir struct to specify when to split/merge shards in a keyspace.

:raft_kv uses the following 3 stats to judge whether splitting/merging should be done:

  1. number of keys in a shard
  2. aggregated data size (in an arbitrary unit) in a shard
  3. current load (computing resource required, in an arbitrary unit) of a shard

Fields:

  • :max_shards Maximum number of shards. If number of shards is the same as this value, no further split occurs.
  • :min_shards Minimum number of shards. If number of shards is the same as this value, no further merge occurs.
  • :max_keys_per_shard Threshold number of keys for shard split. Shards that contains more keys than this value become candidates for split. If nil, shards are not split due to number of keys. Defaults to nil.
  • :max_size_per_shard Threshold size for shard split. Shards whose aggregated size exceeds this value become candidates for split. If nil, shards are not split due to size. Defaults to nil.
  • :max_load_per_shard Threshold load for shard split. Shards that have been experiencing load above this value become candidates for split. If nil, shards are not split due to load. Defaults to nil.
  • :load_per_query_to_missing_key RaftKV.query/4 returns {:error, :key_not_found} if target key does not exist. This value is used as the "load" for such queries. Defaults to 0.
  • :merge_threshold_ratio: For each of the 3 types of split thresholds above (:max_keys_per_shard, :max_size_per_shard and :max_load_per_shard), merge thresholds are calculated by multiplying this ratio. Consecutive 2 shards become candidates for merge if they together (1) contain less keys than the threshold, (2) contain smaller size than the threshold, and (3) experience less load than the threshold. Defaults to 0.5.

Link to this section Summary

Functions

Type-aware getter for load_per_query_to_missing_key.

Type-aware setter for load_per_query_to_missing_key.

Type-aware getter for max_keys_per_shard.

Type-aware setter for max_keys_per_shard.

Type-aware getter for max_load_per_shard.

Type-aware setter for max_load_per_shard.

Type-aware getter for max_shards.

Type-aware setter for max_shards.

Type-aware getter for max_size_per_shard.

Type-aware setter for max_size_per_shard.

Type-aware getter for merge_threshold_ratio.

Type-aware setter for merge_threshold_ratio.

Type-aware getter for min_shards.

Type-aware setter for min_shards.

Creates a new instance of RaftKV.SplitMergePolicy by using the given dict.

A variant of new/1 which returns t or raise if validation fails.

Updates an existing instance of RaftKV.SplitMergePolicy with the given dict. The values in the dict are validated by each field's valid?/1 function. Returns {:ok, valid_struct} or {:error, reason}.

A variant of update/2 which returns t or raise if validation fails. In other words, update/2 followed by Croma.Result.get!/1.

Checks if the given value belongs to t/0 or not.

Link to this section Types

Link to this type

t() View Source
t() :: %RaftKV.SplitMergePolicy{
  load_per_query_to_missing_key: Croma.NonNegInteger.t(),
  max_keys_per_shard: Croma.TypeGen.Nilable.Croma.PosInteger.t(),
  max_load_per_shard: Croma.TypeGen.Nilable.Croma.PosInteger.t(),
  max_shards: Croma.PosInteger.t(),
  max_size_per_shard: Croma.TypeGen.Nilable.Croma.PosInteger.t(),
  merge_threshold_ratio: RaftKV.SplitMergePolicy.MergeThresholdRatio.t(),
  min_shards: Croma.PosInteger.t()
}

Link to this section Functions

Link to this function

load_per_query_to_missing_key(split_merge_policy) View Source
load_per_query_to_missing_key(t()) :: Croma.NonNegInteger.t()

Type-aware getter for load_per_query_to_missing_key.

Link to this function

load_per_query_to_missing_key(s, field) View Source
load_per_query_to_missing_key(t(), Croma.NonNegInteger.t()) :: t()

Type-aware setter for load_per_query_to_missing_key.

Link to this function

max_keys_per_shard(split_merge_policy) View Source
max_keys_per_shard(t()) :: Croma.TypeGen.Nilable.Croma.PosInteger.t()

Type-aware getter for max_keys_per_shard.

Link to this function

max_keys_per_shard(s, field) View Source
max_keys_per_shard(t(), Croma.TypeGen.Nilable.Croma.PosInteger.t()) :: t()

Type-aware setter for max_keys_per_shard.

Link to this function

max_load_per_shard(split_merge_policy) View Source
max_load_per_shard(t()) :: Croma.TypeGen.Nilable.Croma.PosInteger.t()

Type-aware getter for max_load_per_shard.

Link to this function

max_load_per_shard(s, field) View Source
max_load_per_shard(t(), Croma.TypeGen.Nilable.Croma.PosInteger.t()) :: t()

Type-aware setter for max_load_per_shard.

Link to this function

max_shards(split_merge_policy) View Source
max_shards(t()) :: Croma.PosInteger.t()

Type-aware getter for max_shards.

Link to this function

max_shards(s, field) View Source
max_shards(t(), Croma.PosInteger.t()) :: t()

Type-aware setter for max_shards.

Link to this function

max_size_per_shard(split_merge_policy) View Source
max_size_per_shard(t()) :: Croma.TypeGen.Nilable.Croma.PosInteger.t()

Type-aware getter for max_size_per_shard.

Link to this function

max_size_per_shard(s, field) View Source
max_size_per_shard(t(), Croma.TypeGen.Nilable.Croma.PosInteger.t()) :: t()

Type-aware setter for max_size_per_shard.

Link to this function

merge_threshold_ratio(split_merge_policy) View Source
merge_threshold_ratio(t()) :: RaftKV.SplitMergePolicy.MergeThresholdRatio.t()

Type-aware getter for merge_threshold_ratio.

Link to this function

merge_threshold_ratio(s, field) View Source
merge_threshold_ratio(t(), RaftKV.SplitMergePolicy.MergeThresholdRatio.t()) ::
  t()

Type-aware setter for merge_threshold_ratio.

Link to this function

min_shards(split_merge_policy) View Source
min_shards(t()) :: Croma.PosInteger.t()

Type-aware getter for min_shards.

Link to this function

min_shards(s, field) View Source
min_shards(t(), Croma.PosInteger.t()) :: t()

Type-aware setter for min_shards.

Creates a new instance of RaftKV.SplitMergePolicy by using the given dict.

For missing fields, default/0 of each field type will be used.

Returns {:ok, valid_struct} or {:error, reason}. The values in the dict are validated by each field's valid?/1 function.

A variant of new/1 which returns t or raise if validation fails.

In other words, new/1 followed by Croma.Result.get!/1.

Updates an existing instance of RaftKV.SplitMergePolicy with the given dict. The values in the dict are validated by each field's valid?/1 function. Returns {:ok, valid_struct} or {:error, reason}.

Link to this function

update!(s, dict) View Source
update!(t(), Dict.t()) :: t()

A variant of update/2 which returns t or raise if validation fails. In other words, update/2 followed by Croma.Result.get!/1.

Checks if the given value belongs to t/0 or not.