Overview

Module shards_local

• Description
• Data Types
• Function Index
• Function Details

Description

This module implements local or single-node Sharding.

Shards is compatible with ETS API, most of the functions preserves the same contract and semantic, but there are some exceptions (check functions documentation).

Shards provides a logical view of a single ETS table, but internally, that logical table is partitioned in multiple physical ETS tables or Shards, where Shards = [0 .. N-1], and N is the number of shards you want to partition the table.

The K/V pairs are distributed across these shards, therefore, some of the functions don't follow the same semantics as the original ones (analogous ETS functions).

A good example of those are the query-based functions, which return multiple results; in case of ordered_set, with a particular order. For example: select/1,2,3`, `select_reverse/1,2,3, match/1,2,3, match_object/1,2,3, etc.

For those cases, the order which results are returned is not guaranteed to be the same as the original ETS functions.

Additionally to the ETS functions, shards_local module allows to pass an optional argument, the State. When shards is called without the State, it must fetch the state first, and it is recovered doing an extra call to an ETS control table owned by shards_owner_sup. If any microsecond matters, you can skip it call by calling shards_local directly and passing the State. E.g.:

   % create a table
   tab_name = shards:new(tab_name, [{n_shards, 4}]).
  
   % you can get the state at any time by calling:
   State = shards_state:get(tab_name).
  
   % normal way
   shards:lookup(table, key1).
  
   % calling shards_local directly
   shards_local:lookup(table, key1, State).
  
   % if you have created the table with default options,
   % you can skip the sate
   shards_local:lookup(table, key1).

Data Types

access()

access() = public | protected | private

continuation()

continuation() = {Tab::atom(), MatchSpec::ets:match_spec(), Limit::pos_integer(), Shard::non_neg_integer(), Continuation::cont()}

Defines the convention to ets:select/1,3 continuation:

• Tab: Table name.
• MatchSpec: The ets:match_spec().
• Limit: Results limit.
• Shard: Shards number.
• Continuation: The ets:continuation().

filename()

filename() = string() | binary() | atom()

info_item()

info_item() = compressed | fixed | heir | keypos | memory | name | named_table | node | owner | protection | safe_fixed | size | stats | type | write_concurrency | read_concurrency | shards

info_tuple()

info_tuple() = {compressed, boolean()} | {heir, pid() | none} | {keypos, pos_integer()} | {memory, non_neg_integer()} | {name, atom()} | {named_table, boolean()} | {node, node()} | {owner, pid()} | {protection, access()} | {size, non_neg_integer()} | {type, type()} | {write_concurrency, boolean()} | {read_concurrency, boolean()} | {shards, [atom()]}

option()

option() = type() | access() | named_table | {keypos, pos_integer()} | {heir, pid(), HeirData::term()} | {heir, none} | tweaks() | shards_opt()

Create table options – used by new/2.

shards_opt()

shards_opt() = {scope, l | g} | {n_shards, pos_integer()} | {pick_shard_fun, shards_state:pick_fun()} | {pick_node_fun, shards_state:pick_fun()} | {restart_strategy, one_for_one | one_for_all}

Shards extended options.

tabinfo_item()

tabinfo_item() = {name, atom()} | {type, type()} | {protection, access()} | {named_table, boolean()} | {keypos, non_neg_integer()} | {size, non_neg_integer()} | {extended_info, [md5sum | object_count]} | {version, {Major::non_neg_integer(), Minor::non_neg_integer()}} | {shards, [atom()]}

tweaks()

tweaks() = {write_concurrency, boolean()} | {read_concurrency, boolean()} | compressed

ETS tweaks option

type()

type() = set | ordered_set | bag | duplicate_bag

Function Index

Function Details

all/0

Equivalent to ets:all().

delete/1

This operation behaves like ets:delete/1.

See also: ets:delete/1.

delete/2

Equivalent to delete(Tab, Key, shards_state:new()).

delete/3

This operation behaves like ets:delete/2.

See also: ets:delete/2.

delete_all_objects/1

Equivalent to delete_all_objects(Tab, shards_state:new()).

delete_all_objects/2

This operation behaves like ets:delete_all_objects/1.

See also: ets:delete_all_objects/1.

delete_object/2

Equivalent to delete_object(Tab, Object, shards_state:new()).

delete_object/3

This operation behaves like ets:delete_object/2.

See also: ets:delete_object/2.

file2tab/1

Equivalent to file2tab(Filename, []).

file2tab/2

Similar to shards:file2tab/2. Moreover, it restores the supervision tree for the shards corresponding to the given file, such as if they had been created using shards:new/2,3.

See also: ets:file2tab/2.

first/1

Equivalent to first(Tab, shards_state:new()).

first/2

This operation behaves similar to ets:first/1. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:first/1.

foldl/3

Equivalent to foldl(Function, Acc0, Tab, shards_state:new()).

foldl/4

This operation behaves like ets:foldl/3.

See also: ets:foldl/3.

foldr/3

Equivalent to foldr(Function, Acc0, Tab, shards_state:new()).

foldr/4

This operation behaves like ets:foldr/3.

See also: ets:foldr/3.

give_away/3

Equivalent to give_away(Tab, Pid, GiftData, shards_state:new()).

give_away/4

Equivalent to ets:give_away/3 for each shard table. It returns a boolean() instead that just true. Returns true if the function was applied successfully on each shard, otherwise false is returned.

WARNING: It is not recommended execute this function, since it might cause an unexpected behavior. Once this function is executed, shards doesn't control/manage the ETS shards anymore. So from this point, you should use ETS API instead. Also it is recommended to run shards:delete/1 after run this function.

See also: ets:give_away/3.

i/0

Equivalent to ets:i().

info/1

Equivalent to info(Tab, shards_state:new()).

info/2

If 2nd argument is info_item() this function behaves like ets:info/2, but if it is the shards_state:state(), it behaves like ets:info/1.

See also: ets:info/1, ets:info/2.

info/3

This operation is analogous to ets:info/2.

See also: ets:info/2.

info_shard/1

Equivalent to ets:info(ShardTab).

info_shard/2

Equivalent to ets:info(ShardTab, Item).

insert/2

Equivalent to insert(Tab, ObjOrObjs, shards_state:new()).

insert/3

This operation behaves similar to ets:insert_new/2, BUT with a big difference, IT IS NOT ATOMIC, which means, if it fails inserting some object, previous inserted objects are not rolled back, in that case an error is raised.

See also: ets:insert/2.

insert_new/2

Equivalent to insert_new(Tab, ObjOrObjs, shards_state:new()).

insert_new/3

This operation behaves similar to ets:insert_new/2, BUT with a big difference, IT IS NOT ATOMIC, which means if it fails to insert some object, previously inserted ones are rolled back but in a separate command, so there might be race conditions during its execution.

Example:

  > shards:insert_new(mytab, {k1, 1}).
  true
 
  > shards:insert_new(mytab, {k1, 1}).
  false
 
  > shards:insert_new(mytab, [{k1, 1}, {k2, 2}]).
  false

See also: ets:insert_new/2.

is_compiled_ms/1

Equivalent to ets:is_compiled_ms(Term).

last/1

Equivalent to last(Tab, shards_state:new()).

last/2

This operation behaves similar to ets:last/1. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:last/1.

lookup/2

Equivalent to lookup(Tab, Key, shards_state:new()).

lookup/3

This operation behaves like ets:lookup/2.

See also: ets:lookup/2.

lookup_element/3

Equivalent to lookup_element(Tab, Key, Pos, shards_state:new()).

lookup_element/4

This operation behaves like ets:lookup_element/3.

See also: ets:lookup_element/3.

match/1

This operation behaves similar to ets:match/1. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:match/1.

match/2

match/3

If 3rd argument is pos_integer() this function behaves like ets:match/3, but if it is the shards_state:state(), it behaves like ets:match/2.

See also: ets:match/2, ets:match/3.

match/4

This operation behaves similar to ets:match/3. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:match/3.

match_delete/2

Equivalent to match_delete(Tab, Pattern, shards_state:new()).

match_delete/3

This operation behaves like ets:match_delete/2.

See also: ets:match_delete/2.

match_object/1

This operation behaves similar to ets:match_object/1. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:match_object/1.

match_object/2

Equivalent to match_object(Tab, Pattern, shards_state:new()).

match_object/3

If 3rd argument is pos_integer() this function behaves like ets:match_object/3, but if it is the shards_state:state(), it behaves like ets:match_object/2.

See also: ets:match_object/2, ets:match_object/3.

match_object/4

This operation behaves similar to ets:match_object/3. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:match_object/3.

match_spec_compile/1

Equivalent to ets:match_spec_compile(MatchSpec).

match_spec_run/2

Equivalent to ets:match_spec_run(List, CompiledMatchSpec).

member/2

Equivalent to member(Tab, Key, shards_state:new()).

member/3

This operation behaves like ets:member/2.

See also: ets:member/2.

new/2

This operation is analogous to ets:new/2, BUT it behaves totally different. When this function is called, instead of create a single table, a new supervision tree is created and added to shards_sup.

This supervision tree has a main supervisor shards_sup which creates a control ETS table and also creates N number of shards_owner (being N the number of shards). Each shards_owner creates an ETS table to represent each shard, so this gen_server acts as the table owner.

Finally, when you create a table, internally N physical tables are created (one per shard), but shards encapsulates all this and you see only one logical table (similar to how a distributed storage works).

See also: ets:new/2.

next/2

Equivalent to next(Tab, Key1, shards_state:new()).

next/3

This operation behaves similar to ets:next/2. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning. It raises a bad_pick_fun_ret exception in case of pick fun returns any.

See also: ets:next/2.

prev/2

Equivalent to prev(Tab, Key1, shards_state:new()).

prev/3

This operation behaves similar to ets:prev/2. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:prev/2.

rename/2

Equivalent to rename(Tab, Name, shards_state:new()).

rename/3

Equivalent to ets:rename/2.

See also: ets:rename/2.

safe_fixtable/2

Equivalent to safe_fixtable(Tab, Fix, shards_state:new()).

safe_fixtable/3

Equivalent to ets:safe_fixtable/2 for each shard table. It returns a boolean() instead that just true. Returns true if the function was applied successfully on each shard, otherwise false is returned.

See also: ets:safe_fixtable/2.

select/1

This operation behaves similar to ets:select/1. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:select/1.

select/2

Equivalent to select(Tab, MatchSpec, shards_state:new()).

select/3

If 3rd argument is pos_integer() this function behaves like ets:select/3, but if it is the shards_state:state(), it behaves like ets:select/2.

See also: ets:select/2, ets:select/3.

select/4

This operation behaves similar to ets:select/3. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:select/3.

select_count/2

Equivalent to select_count(Tab, MatchSpec, shards_state:new()).

select_count/3

This operation behaves like ets:select_count/2.

See also: ets:select_count/2.

select_delete/2

Equivalent to select_delete(Tab, MatchSpec, shards_state:new()).

select_delete/3

This operation behaves like ets:select_delete/2.

See also: ets:select_delete/2.

select_reverse/1

This operation behaves similar to ets:select_reverse/1. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:select_reverse/1.

select_reverse/2

Equivalent to select_reverse(Tab, MatchSpec, shards_state:new()).

select_reverse/3

If 3rd argument is pos_integer() this function behaves like ets:select_reverse/3, but if it is the shards_state:state(), it behaves like ets:select_reverse/2.

See also: ets:select_reverse/2, ets:select_reverse/3.

select_reverse/4

This operation behaves similar to ets:select_reverse/3. The order in which results are returned, might be not the same as the original ETS function. Remember shards architecture described at the beginning.

See also: ets:select_reverse/3.

setopts/2

Equivalent to setopts(Tab, Opts, shards_state:new()).

setopts/3

Equivalent to ets:setopts/2 for each shard table. It returns a boolean() instead that just true. Returns true if the function was applied successfully on each shard, otherwise false is returned.

See also: ets:setopts/2.

tab2file/2

Equivalent to tab2file(Tab, Filenames, shards_state:new()).

tab2file/3

Equivalent to tab2file / 4.

tab2file/4

Similar to ets:tab2file/3, but it behaves different. This function generates one file per shard using ets:tab2file/3, and also generate a master file with the given Filename that holds the information about the other shards files in order to be able to recover them using ets:file2tab/1,2.

See also: ets:tab2file/3.

tab2list/1

Equivalent to tab2list(Tab, shards_state:new()).

tab2list/2

This operation behaves like ets:tab2list/1.

See also: ets:tab2list/1.

tabfile_info/1

This operation is analogous to ets:tabfile_info/1, but it adds info about the shards {shards, [atom()]}.

See also: ets:tabfile_info/1.

table/1

Equivalent to table(Tab, shards_state:new()).

table/2

Equivalent to table / 3.

table/3

Similar to ets:table/2, but it returns a list of ets:table/2 responses, one for each shard table.

See also: ets:table/2.

take/2

Equivalent to take(Tab, Key, shards_state:new()).

take/3

This operation behaves like ets:take/2.

See also: ets:take/2.

test_ms/2

Equivalent to ets:test_ms(Tuple, MatchSpec).

update_counter/3

Equivalent to update_counter(Tab, Key, UpdateOp, shards_state:new()).

update_counter/4

This operation behaves like ets:update_counter/3.

See also: ets:update_counter/3.

update_counter/5

This operation behaves like ets:update_counter/4.

See also: ets:update_counter/4.

update_element/3

Equivalent to update_element(Tab, Key, ElementSpec, shards_state:new()).

update_element/4

This operation behaves like ets:update_element/3.

See also: ets:update_element/3.

Overview

Generated by EDoc