View Source shards (shards v1.1.1)

Partitioned/Sharded ETS tables.

Features

shards implements the ETS API to keep compatibility and make the usage straightforward; exactly the same if you were using ets.
Sharded or partitioned tables under-the-hood. This feature is managed entirely by shards and it is 100% transparent for the client. Provides a logical view of a single ETS table, but internally, it is composed by N number of partitions and each partition has associated an ETS table.
High performance and scalability. shards keeps the lock contention under control enabling ETS tables to scale out and support higher levels of concurrency without lock issues; specially write-locks, which most of the cases might cause significant performance degradation.

Partitioned Table

When a table is created with shards:new/2, a new supervision tree is created to represent the partitioned table. There is a main supervisor shards_partition_sup that create an ETS table to store the metadata and also starts the children which are the partitions to create. Each partition is owned by shards_partition (it is a gen_server) and it creates an ETS table for storing data mapped to that partition. The supervision tree looks like:

                             [shards_partition]--><ETS-Table>
                             /
   [shards_partition_sup]--<-[shards_partition]--><ETS-Table>
             |               \
    <Metadata-ETS-Table>     [shards_partition]--><ETS-Table>

The returned value by shards:new/2 may be an atom if it is a named table or a reference otherwise, and in the second case the returned reference is the one of the metadata table, which is the main entry point and it is owned by the main supervisor. See shards:new/2 for more information.

Usage

   > Tab = shards:new(tab, []).
   #Ref<0.1541908042.2337144842.31535>
   > shards:insert(Tab, [{a, 1}, {b, 2}, {c, 3}]).
   true
   > shards:lookup(Tab, a).
   [{a,1}]
   > shards:lookup(Tab, b).
   [{b,2}]
   > shards:lookup(Tab, c).
   [{c,3}]
   > shards:lookup_element(Tab, c, 2).
   3
   > shards:lookup(Tab, d).
   []
   > shards:delete(Tab, c).
   true
   > shards:lookup(Tab, c).
   []

As you can see, the usage is exactly the same if you were using ets, you can try the rest of the ETS API but with shards module.

Important

Despite shards aims to keep 100% compatibility with current ETS API, the semantic for some of the functions may be a bit different due to the nature of sharding; it is not the same having all the entries in a single table than distributed across multiple ones. For example, for query-based functions like select, match, etc., the returned entries are the same but not necessary the same order than ets. For first, next, and last they behave the similar in the sense by means of them a partitioned table is traversed, so the final result is the same, but the order in which the entries are traversed may be different. Therefore, it is highly recommended to read the documentation of the functions.

Summary

Types

access/0

continuation/0

ets_continuation/0

filename/0

info_item/0

info_tuple/0

option/0

shards_opt/0

tab/0

tabinfo_item/0

tweaks/0

type/0

Functions

all()

Equivalent to ets:all().

delete(Tab)

Equivalent to ets:delete/1.

See also: ets:delete/1.

delete(Tab, Key)

Equivalent to delete(Tab, Key, shards_meta:get(Tab)).

delete(Tab, Key, Meta)

Equivalent to ets:delete/2.

See also: ets:delete/2.

delete_all_objects(Tab)

Equivalent to delete_all_objects(Tab, shards_meta:get(Tab)).

delete_all_objects(Tab, Meta)

Equivalent to ets:delete_all_objects/1.

See also: ets:delete_all_objects/1.

delete_object(Tab, Object)

Equivalent to delete_object(Tab, Object, shards_meta:get(Tab)).

delete_object(Tab, Object, Meta)

Equivalent to ets:delete_object/2.

See also: ets:delete_object/2.

file2tab(Filename)

Equivalent to file2tab(Filename, []).

file2tab(Filename, Options)

Equivalent to shards:file2tab/2. Moreover, it restores the supervision tree for the shards corresponding to the given file, such as if they had been created using shards:new/2.

See also: ets:file2tab/2.

first(Tab)

Equivalent to first(Tab, shards_meta:get(Tab)).

first(Tab, Meta)

Equivalent to ets:first/1.

foldl(Fun, Acc, Tab)

Equivalent to foldl(Fun, Acc, Tab, shards_meta:get(Tab)).

foldl(Fun, Acc0, Tab, Meta)

Equivalent to ets:foldl/3.

foldr(Fun, Acc, Tab)

Equivalent to foldr(Fun, Acc, Tab, shards_meta:get(Tab)).

foldr(Fun, Acc0, Tab, Meta)

Equivalent to ets:foldr/3.

get_meta(Tab, Key)

Equivalent to get_meta(Tab, Key, undefined).

get_meta(Tab, Key, Def)

Wrapper for shards_meta:get/3.

i()

Equivalent to ets:i().

info(Tab)

Similar to ets:info/1' but extra information about the partitioned table is added.

info(Tab, Item)

Equivalent to ets:info/2.

insert(Tab, ObjOrObjs)

Equivalent to insert(Tab, ObjOrObjs, shards_meta:get(Tab)).

insert(Tab, ObjOrObjs, Meta)

Equivalent to ets:insert/2.

insert_new(Tab, ObjOrObjs)

Equivalent to insert_new(Tab, ObjOrObjs, shards_meta:get(Tab)).

insert_new(Tab, ObjOrObjs, Meta)

Equivalent to ets:insert_new/2.

is_compiled_ms(Term)

Equivalent to ets:is_compiled_ms(Term).

last(Tab)

Equivalent to ets:last/1.

lookup(Tab, Key)

Equivalent to lookup(Tab, Key, shards_meta:get(Tab)).

lookup(Tab, Key, Meta)

Equivalent to ets:lookup/2.

See also: ets:lookup/2.

lookup_element(Tab, Key, Pos)

Equivalent to lookup_element(Tab, Key, Pos, shards_meta:get(Tab)).

lookup_element(Tab, Key, Pos, Meta)

Equivalent to ets:lookup_element/3.

See also: ets:lookup_element/3.

match(Continuation)

Equivalent to ets:match/1.

match(Tab, Pattern)

Equivalent to match(Tab, Pattern, shards_meta:get(Tab)).

match(Tab, Pattern, LimitOrMeta)

If 3rd argument is pos_integer() this function behaves like ets:match/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:match/2.

match(Tab, Pattern, Limit, Meta)

Equivalent to ets:match/3.

match_delete(Tab, Pattern)

Equivalent to match_delete(Tab, Pattern, shards_meta:get(Tab)).

match_delete(Tab, Pattern, Meta)

Equivalent to ets:match_delete/2.

See also: ets:match_delete/2.

match_object(Cont)

Equivalent to ets:match_object/1.

match_object(Tab, Pattern)

Equivalent to match_object(Tab, Pattern, shards_meta:get(Tab)).

match_object(Tab, Pattern, LimitOrMeta)

If 3rd argument is pos_integer() this function behaves like ets:match_object/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:match_object/2.

match_object(Tab, Pattern, Limit, Meta)

Equivalent to ets:match_object/3.

match_spec_compile(MatchSpec)

Equivalent to ets:match_spec_compile(MatchSpec).

match_spec_run(List, CompiledMatchSpec)

Equivalent to ets:match_spec_run(List, CompiledMatchSpec).

member(Tab, Key)

Equivalent to member(Tab, Key, shards_meta:get(Tab)).

member(Tab, Key, Meta)

Equivalent to ets:member/2.

See also: ets:member/2.

new(Name, Options)

This operation is equivalent to ets:new/2, but when is called, instead of create a single ETS table, it creates a new supervision tree for the partitioned table.

next(Tab, Key1, Meta)

Equivalent to ets:next/2.

partition_owners(TabOrPid)

Returns the partition PIDs associated with the given table TabOrPid.

prev(Tab, Key1)

Equivalent to ets:next/2.

put_meta(Tab, Key, Val)

Wrapper for shards_meta:put/3.

rename(Tab, Name)

Equivalent to rename(Tab, Name, shards_meta:get(Tab)).

rename(Tab, Name, Meta)

Equivalent to ets:rename/2.

safe_fixtable(Tab, Fix)

Equivalent to safe_fixtable(Tab, Fix, shards_meta:get(Tab)).

select(Cont)

Equivalent to ets:select/1.

select(Tab, MatchSpec)

Equivalent to select(Tab, MatchSpec, shards_meta:get(Tab)).

select(Tab, MatchSpec, LimitOrMeta)

If 3rd argument is pos_integer() this function behaves like ets:select/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:select/2.

select(Tab, MatchSpec, Limit, Meta)

Equivalent to ets:select/3.

select_count(Tab, MatchSpec)

Equivalent to select_count(Tab, MatchSpec, shards_meta:get(Tab)).

select_count(Tab, MatchSpec, Meta)

Equivalent to ets:select_count/2.

See also: ets:select_count/2.

select_delete(Tab, MatchSpec)

Equivalent to select_delete(Tab, MatchSpec, shards_meta:get(Tab)).

select_delete(Tab, MatchSpec, Meta)

Equivalent to ets:select_delete/2.

See also: ets:select_delete/2.

select_replace(Tab, MatchSpec)

Equivalent to select_replace(Tab, MatchSpec, shards_meta:get(Tab)).

select_replace(Tab, MatchSpec, Meta)

Equivalent to ets:select_replace/2.

See also: ets:select_replace/2.

select_reverse(Cont)

Equivalent to ets:select_reverse/1.

select_reverse(Tab, MatchSpec)

Equivalent to select_reverse(Tab, MatchSpec, shards_meta:get(Tab)).

select_reverse(Tab, MatchSpec, LimitOrMeta)

If 3rd argument is pos_integer() this function behaves like ets:select_reverse/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:select_reverse/2.

select_reverse(Tab, MatchSpec, Limit, Meta)

Equivalent to ets:select_reverse/3.

setopts(Tab, Opts)

Equivalent to setopts(Tab, Opts, shards_meta:get(Tab)).

setopts(Tab, Opts, Meta)

Equivalent to ets:setopts/2.

tab2file(Tab, Filename)

Equivalent to tab2file(Tab, Filename, []).

tab2file(Tab, Filename, Options)

Equivalent to ets:tab2file/3.

tab2list(Tab)

Equivalent to tab2list(Tab, shards_meta:get(Tab)).

tab2list(Tab, Meta)

Equivalent to ets:tab2list/1.

See also: ets:tab2list/1.

tabfile_info(Filename)

Equivalent to ets:tabfile_info/1.

table(Tab)

Equivalent to table(Tab, []).

table(Tab, Options)

Equivalent to table(Tab, Options, shards_meta:get(Tab)).

table(Tab, Options, Meta)

Similar to ets:table/2, but it returns a list of qlc:query_handle(); one per partition.

See also: ets:table/2.

table_meta(Tab)

Returns the metadata associated with the given table Tab.

take(Tab, Key)

Equivalent to take(Tab, Key, shards_meta:get(Tab)).

take(Tab, Key, Meta)

Equivalent to ets:take/2.

See also: ets:take/2.

test_ms(Tuple, MatchSpec)

Equivalent to ets:test_ms(Tuple, MatchSpec).

update_counter(Tab, Key, UpdateOp)

Equivalent to update_counter(Tab, Key, UpdateOp, shards_meta:get(Tab)).

update_counter(Tab, Key, UpdateOp, DefaultOrMeta)

Equivalent to ets:update_counter/4.

update_counter(Tab, Key, UpdateOp, Default, Meta)

Equivalent to ets:update_counter/4.

See also: ets:update_counter/4.

update_element(Tab, Key, ElementSpec)

Equivalent to update_element(Tab, Key, ElementSpec, shards_meta:get(Tab)).

update_element(Tab, Key, ElementSpec, Meta)

Equivalent to ets:update_element/3.

See also: ets:update_element/3.

Types

access/0

-type access() :: public | protected | private.

continuation/0

-type continuation() ::
          {Tab :: tab(),
           MatchSpec :: ets:match_spec(),
           Limit :: pos_integer(),
           Partition :: non_neg_integer(),
           Continuation :: ets_continuation()}.

ets_continuation/0

-type ets_continuation() ::
          '$end_of_table' |
          {tab(), integer(), integer(), ets:comp_match_spec(), list(), integer()} |
          {tab(), _, _, integer(), ets:comp_match_spec(), list(), integer(), integer()}.

filename/0

-type filename() :: string() | binary() | atom().

info_item/0

-type info_item() ::
          compressed | fixed | heir | keypos | memory | name | named_table | node | owner | protection |
          safe_fixed | size | stats | type | write_concurrency | read_concurrency | shards.

info_tuple/0

-type info_tuple() ::
          {compressed, boolean()} |
          {heir, pid() | none} |
          {keypos, pos_integer()} |
          {memory, non_neg_integer()} |
          {name, atom()} |
          {named_table, boolean()} |
          {node, node()} |
          {owner, pid()} |
          {protection, access()} |
          {size, non_neg_integer()} |
          {type, type()} |
          {write_concurrency, boolean()} |
          {read_concurrency, boolean()} |
          {shards, [atom()]}.

option/0

-type option() ::
          type() |
          access() |
          named_table |
          {keypos, pos_integer()} |
          {heir, pid(), HeirData :: term()} |
          {heir, none} |
          tweaks() |
          shards_opt().

shards_opt/0

-type shards_opt() ::
          {partitions, pos_integer()} |
          {keyslot_fun, shards_meta:keyslot_fun()} |
          {restore, term(), term()}.

tab/0

-type tab() :: atom() | ets:tid().

tabinfo_item/0

-type tabinfo_item() ::
          {name, atom()} |
          {type, type()} |
          {protection, access()} |
          {named_table, boolean()} |
          {keypos, non_neg_integer()} |
          {size, non_neg_integer()} |
          {extended_info, [md5sum | object_count]} |
          {version, {Major :: non_neg_integer(), Minor :: non_neg_integer()}} |
          {shards, [atom()]}.

tweaks/0

-type tweaks() :: {write_concurrency, boolean()} | {read_concurrency, boolean()} | compressed.

type/0

-type type() :: set | ordered_set | bag | duplicate_bag.

Functions

all()

Equivalent to ets:all().

delete(Tab)

-spec delete(Tab :: tab()) -> true.

Equivalent to ets:delete/1.

See also: ets:delete/1.

delete(Tab, Key)

Equivalent to delete(Tab, Key, shards_meta:get(Tab)).

delete(Tab, Key, Meta)

-spec delete(Tab, Key, Meta) -> true when Tab :: tab(), Key :: term(), Meta :: shards_meta:t().

Equivalent to ets:delete/2.

See also: ets:delete/2.

delete_all_objects(Tab)

Equivalent to delete_all_objects(Tab, shards_meta:get(Tab)).

delete_all_objects(Tab, Meta)

-spec delete_all_objects(Tab, Meta) -> true when Tab :: tab(), Meta :: shards_meta:t().

Equivalent to ets:delete_all_objects/1.

See also: ets:delete_all_objects/1.

delete_object(Tab, Object)

Equivalent to delete_object(Tab, Object, shards_meta:get(Tab)).

delete_object(Tab, Object, Meta)

-spec delete_object(Tab, Object, Meta) -> true
                       when Tab :: tab(), Object :: tuple(), Meta :: shards_meta:t().

Equivalent to ets:delete_object/2.

See also: ets:delete_object/2.

file2tab(Filename)

Equivalent to file2tab(Filename, []).

file2tab(Filename, Options)

-spec file2tab(Filename, Options) -> {ok, Tab} | {error, Reason}
                  when
                      Filename :: filename(),
                      Tab :: tab(),
                      Options :: [Option],
                      Option :: {verify, boolean()},
                      Reason :: term().

Equivalent to shards:file2tab/2. Moreover, it restores the supervision tree for the shards corresponding to the given file, such as if they had been created using shards:new/2.

See also: ets:file2tab/2.

first(Tab)

Equivalent to first(Tab, shards_meta:get(Tab)).

first(Tab, Meta)

-spec first(Tab, Meta) -> Key | '$end_of_table'
               when Tab :: tab(), Key :: term(), Meta :: shards_meta:t().

Equivalent to ets:first/1.

However, the order in which results are returned might be not the same as the original ETS function, since it is a sharded table.

See also: ets:first/1.

foldl(Fun, Acc, Tab)

Equivalent to foldl(Fun, Acc, Tab, shards_meta:get(Tab)).

foldl(Fun, Acc0, Tab, Meta)

-spec foldl(Fun, Acc0, Tab, Meta) -> Acc1
               when
                   Fun :: fun((Element :: term(), AccIn) -> AccOut),
                   Tab :: tab(),
                   Meta :: shards_meta:t(),
                   Acc0 :: term(),
                   Acc1 :: term(),
                   AccIn :: term(),
                   AccOut :: term().

Equivalent to ets:foldl/3.

However, the order in which the entries are traversed may be different since they are distributed across multiple partitions.

See also: ets:foldl/3.

foldr(Fun, Acc, Tab)

Equivalent to foldr(Fun, Acc, Tab, shards_meta:get(Tab)).

foldr(Fun, Acc0, Tab, Meta)

-spec foldr(Fun, Acc0, Tab, Meta) -> Acc1
               when
                   Fun :: fun((Element :: term(), AccIn) -> AccOut),
                   Tab :: tab(),
                   Meta :: shards_meta:t(),
                   Acc0 :: term(),
                   Acc1 :: term(),
                   AccIn :: term(),
                   AccOut :: term().

Equivalent to ets:foldr/3.

However, the order in which the entries are traversed may be different since they are distributed across multiple partitions.

See also: ets:foldr/3.

get_meta(Tab, Key)

Equivalent to get_meta(Tab, Key, undefined).

get_meta(Tab, Key, Def)

-spec get_meta(Tab, Key, Def) -> Val when Tab :: tab(), Key :: term(), Def :: term(), Val :: term().

Wrapper for shards_meta:get/3.

i()

Equivalent to ets:i().

info(Tab)

-spec info(Tab) -> InfoList | undefined
              when Tab :: tab(), InfoList :: [InfoTuple], InfoTuple :: info_tuple().

Similar to ets:info/1' but extra information about the partitioned table is added.

Extra Info:

{partitions, post_integer()} - Number of partitions.
{keyslot_fun, shards_meta:keyslot_fun()} - Functions used for compute the keyslot.
{parallel, boolean()} - Whether the parallel mode is enabled or not.

See also: ets:info/1.

info(Tab, Item)

-spec info(Tab, Item) -> Value | undefined when Tab :: tab(), Item :: info_item(), Value :: term().

Equivalent to ets:info/2.

See the added items by shards:info/1.

See also: ets:info/2.

insert(Tab, ObjOrObjs)

Equivalent to insert(Tab, ObjOrObjs, shards_meta:get(Tab)).

insert(Tab, ObjOrObjs, Meta)

-spec insert(Tab, ObjOrObjs, Meta) -> true | no_return()
                when Tab :: tab(), ObjOrObjs :: tuple() | [tuple()], Meta :: shards_meta:t().

Equivalent to ets:insert/2.

Despite this functions behaves exactly the same as ets:insert/2 and produces the same result, there is a big difference due to the nature of the sharding distribution model, IT IS NOT ATOMIC. Therefore, if it fails by inserting an object at some partition, previous inserts execution on other partitions are not rolled back, but an error is raised instead.

See also: ets:insert/2.

insert_new(Tab, ObjOrObjs)

Equivalent to insert_new(Tab, ObjOrObjs, shards_meta:get(Tab)).

insert_new(Tab, ObjOrObjs, Meta)

-spec insert_new(Tab, ObjOrObjs, Meta) -> boolean()
                    when Tab :: tab(), ObjOrObjs :: tuple() | [tuple()], Meta :: shards_meta:t().

Equivalent to ets:insert_new/2.

Despite this functions behaves exactly the same as ets:insert_new/2 and produces the same result, there is a big difference due to the nature of the sharding distribution model, IT IS NOT ATOMIC. Opposite to shards:insert/2, this function tries to roll-back previous inserts execution on other partitions if it fails by inserting an object at some partition, but there might be race conditions during roll-back execution.

Example:

  > shards:insert_new(mytab, {k1, 1}).
  true
 
  > shards:insert_new(mytab, {k1, 1}).
  false
 
  > shards:insert_new(mytab, [{k1, 1}, {k2, 2}]).
  false

See also: ets:insert_new/2.

is_compiled_ms(Term)

Equivalent to ets:is_compiled_ms(Term).

last(Tab)

-spec last(Tab) -> Key | '$end_of_table' when Tab :: tab(), Key :: term().

Equivalent to ets:last/1.

However, the order in which results are returned might be not the same as the original ETS function, since it is a sharded table.

See also: ets:last/1.

lookup(Tab, Key)

Equivalent to lookup(Tab, Key, shards_meta:get(Tab)).

lookup(Tab, Key, Meta)

-spec lookup(Tab, Key, Meta) -> [Object]
                when Tab :: tab(), Key :: term(), Meta :: shards_meta:t(), Object :: tuple().

Equivalent to ets:lookup/2.

See also: ets:lookup/2.

lookup_element(Tab, Key, Pos)

Equivalent to lookup_element(Tab, Key, Pos, shards_meta:get(Tab)).

lookup_element(Tab, Key, Pos, Meta)

-spec lookup_element(Tab, Key, Pos, Meta) -> Elem
                        when
                            Tab :: tab(),
                            Key :: term(),
                            Pos :: pos_integer(),
                            Meta :: shards_meta:t(),
                            Elem :: term() | [term()].

Equivalent to ets:lookup_element/3.

See also: ets:lookup_element/3.

match(Continuation)

-spec match(Continuation) -> {[Match], continuation()} | '$end_of_table'
               when Match :: [term()], Continuation :: continuation().

Equivalent to ets:match/1.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:match/1.

match(Tab, Pattern)

Equivalent to match(Tab, Pattern, shards_meta:get(Tab)).

match(Tab, Pattern, LimitOrMeta)

-spec match(Tab, Pattern, LimitOrMeta) -> {[Match], Cont} | '$end_of_table' | [Match]
               when
                   Tab :: tab(),
                   Pattern :: ets:match_pattern(),
                   LimitOrMeta :: pos_integer() | shards_meta:t(),
                   Match :: [term()],
                   Cont :: continuation().

If 3rd argument is pos_integer() this function behaves like ets:match/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:match/2.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:match/2, ets:match/3.

match(Tab, Pattern, Limit, Meta)

-spec match(Tab, Pattern, Limit, Meta) -> {[Match], Cont} | '$end_of_table'
               when
                   Tab :: tab(),
                   Pattern :: ets:match_pattern(),
                   Limit :: pos_integer(),
                   Meta :: shards_meta:t(),
                   Match :: [term()],
                   Cont :: continuation().

Equivalent to ets:match/3.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:match/3.

match_delete(Tab, Pattern)

Equivalent to match_delete(Tab, Pattern, shards_meta:get(Tab)).

match_delete(Tab, Pattern, Meta)

-spec match_delete(Tab, Pattern, Meta) -> true
                      when Tab :: tab(), Pattern :: ets:match_pattern(), Meta :: shards_meta:t().

Equivalent to ets:match_delete/2.

See also: ets:match_delete/2.

match_object(Cont)

-spec match_object(Cont) -> {[Object], Cont} | '$end_of_table'
                      when Object :: tuple(), Cont :: continuation().

Equivalent to ets:match_object/1.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:match_object/1.

match_object(Tab, Pattern)

Equivalent to match_object(Tab, Pattern, shards_meta:get(Tab)).

match_object(Tab, Pattern, LimitOrMeta)

-spec match_object(Tab, Pattern, LimitOrMeta) -> {[Object], Cont} | '$end_of_table' | [Object]
                      when
                          Tab :: tab(),
                          Pattern :: ets:match_pattern(),
                          LimitOrMeta :: pos_integer() | shards_meta:t(),
                          Object :: tuple(),
                          Cont :: continuation().

If 3rd argument is pos_integer() this function behaves like ets:match_object/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:match_object/2.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:match_object/3.

match_object(Tab, Pattern, Limit, Meta)

-spec match_object(Tab, Pattern, Limit, Meta) -> {[Object], Cont} | '$end_of_table'
                      when
                          Tab :: tab(),
                          Pattern :: ets:match_pattern(),
                          Limit :: pos_integer(),
                          Meta :: shards_meta:t(),
                          Object :: tuple(),
                          Cont :: continuation().

Equivalent to ets:match_object/3.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:match_object/3.

match_spec_compile(MatchSpec)

Equivalent to ets:match_spec_compile(MatchSpec).

match_spec_run(List, CompiledMatchSpec)

Equivalent to ets:match_spec_run(List, CompiledMatchSpec).

member(Tab, Key)

Equivalent to member(Tab, Key, shards_meta:get(Tab)).

member(Tab, Key, Meta)

-spec member(Tab, Key, Meta) -> boolean() when Tab :: tab(), Key :: term(), Meta :: shards_meta:t().

Equivalent to ets:member/2.

See also: ets:member/2.

new(Name, Options)

-spec new(Name, Options) -> Tab when Name :: atom(), Options :: [option()], Tab :: tab().

This operation is equivalent to ets:new/2, but when is called, instead of create a single ETS table, it creates a new supervision tree for the partitioned table.

The supervision tree is composed by a main supervisor shards_partition_sup and N number of workers or partitions handled by shards_partition (partition owner). Each worker creates an ETS table to handle the partition. Also, the main supervisor shards_partition_sup creates an ETS table to keep the metadata for the partitioned table.

Returns an atom if the created table is a named table, otherwise, a reference is returned. In the last case, the returned reference is the one of the metadata table, which is the main entry-point and it is owned by the main supervisor shards_partition_sup.

Options:

In addition to the options given by ets:new/2, this functions provides the next options:

{partitions, N} - Specifies the number of partitions for the sharded table. By default, N = erlang:system_info(schedulers_online).
{keyslot_fun, F} - Specifies the function used to compute the partition where the action will be evaluated. Defaults to erlang:phash2/2.
{parallel, P} - Specifies whether shards should work in parallel mode or not, for the applicable functions, e.g.: select, match, etc. By default is set to false.
{parallel_timeout, T} - When parallel is set to true, it specifies the max timeout for a parallel execution. Defaults to infinity.

Access:

Currently, only public access is supported by shards:new/2. Since a partitioned table is started with its own supervision tree when created, it is very tricky to provide private or protected access since there are multiple partitions (or ETS tables) and they are owned by the supervisor's children, and the supervisor along with their children (or partitions) are managed by shards under-the-hood; it is completely transparent for the client.

Examples:

  > Tab = shards:new(tab1, []).
  #Ref<0.1541908042.2337144842.31535>
 
  > shards:new(tab2, [named_table]).
  tab2

See also the "Partitioned Table" section at the module documentation for more information.

See also: ets:new/2.

next(Tab, Key1, Meta)

-spec next(Tab, Key1, Meta) -> Key2 | '$end_of_table'
              when Tab :: tab(), Key1 :: term(), Key2 :: term(), Meta :: shards_meta:t().

Equivalent to ets:next/2.

However, the order in which results are returned might be not the same as the original ETS function, since it is a sharded table.

See also: ets:next/2.

partition_owners(TabOrPid)

-spec partition_owners(TabOrPid) -> [OwnerPid] when TabOrPid :: pid() | tab(), OwnerPid :: pid().

Returns the partition PIDs associated with the given table TabOrPid.

prev(Tab, Key1)

-spec prev(Tab, Key1) -> Key2 | '$end_of_table' when Tab :: tab(), Key1 :: term(), Key2 :: term().

Equivalent to ets:next/2.

However, the order in which results are returned might be not the same as the original ETS function, since it is a sharded table.

See also: ets:prev/2.

put_meta(Tab, Key, Val)

-spec put_meta(Tab, Key, Val) -> ok when Tab :: shards:tab(), Key :: term(), Val :: term().

Wrapper for shards_meta:put/3.

rename(Tab, Name)

Equivalent to rename(Tab, Name, shards_meta:get(Tab)).

rename(Tab, Name, Meta)

-spec rename(Tab, Name, Meta) -> Name when Tab :: tab(), Name :: atom(), Meta :: shards_meta:t().

Equivalent to ets:rename/2.

Renames the table name and all its associated shard tables. If something unexpected occurs during the process, an exception will be raised.

See also: ets:rename/2.

safe_fixtable(Tab, Fix)

Equivalent to safe_fixtable(Tab, Fix, shards_meta:get(Tab)).

select(Cont)

-spec select(Cont) -> {[Match], Cont} | '$end_of_table' when Match :: term(), Cont :: continuation().

Equivalent to ets:select/1.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:select/1.

select(Tab, MatchSpec)

Equivalent to select(Tab, MatchSpec, shards_meta:get(Tab)).

select(Tab, MatchSpec, LimitOrMeta)

-spec select(Tab, MatchSpec, LimitOrMeta) -> {[Match], Cont} | '$end_of_table' | [Match]
                when
                    Tab :: tab(),
                    MatchSpec :: ets:match_spec(),
                    LimitOrMeta :: pos_integer() | shards_meta:t(),
                    Match :: term(),
                    Cont :: continuation().

If 3rd argument is pos_integer() this function behaves like ets:select/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:select/2.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:select/3.

select(Tab, MatchSpec, Limit, Meta)

-spec select(Tab, MatchSpec, Limit, Meta) -> {[Match], Cont} | '$end_of_table'
                when
                    Tab :: tab(),
                    MatchSpec :: ets:match_spec(),
                    Limit :: pos_integer(),
                    Meta :: shards_meta:t(),
                    Match :: term(),
                    Cont :: continuation().

Equivalent to ets:select/3.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:select/3.

select_count(Tab, MatchSpec)

Equivalent to select_count(Tab, MatchSpec, shards_meta:get(Tab)).

select_count(Tab, MatchSpec, Meta)

-spec select_count(Tab, MatchSpec, Meta) -> NumMatched
                      when
                          Tab :: tab(),
                          MatchSpec :: ets:match_spec(),
                          Meta :: shards_meta:t(),
                          NumMatched :: non_neg_integer().

Equivalent to ets:select_count/2.

See also: ets:select_count/2.

select_delete(Tab, MatchSpec)

Equivalent to select_delete(Tab, MatchSpec, shards_meta:get(Tab)).

select_delete(Tab, MatchSpec, Meta)

-spec select_delete(Tab, MatchSpec, Meta) -> NumDeleted
                       when
                           Tab :: tab(),
                           MatchSpec :: ets:match_spec(),
                           Meta :: shards_meta:t(),
                           NumDeleted :: non_neg_integer().

Equivalent to ets:select_delete/2.

See also: ets:select_delete/2.

select_replace(Tab, MatchSpec)

Equivalent to select_replace(Tab, MatchSpec, shards_meta:get(Tab)).

select_replace(Tab, MatchSpec, Meta)

-spec select_replace(Tab, MatchSpec, Meta) -> NumReplaced
                        when
                            Tab :: tab(),
                            MatchSpec :: ets:match_spec(),
                            Meta :: shards_meta:t(),
                            NumReplaced :: non_neg_integer().

Equivalent to ets:select_replace/2.

See also: ets:select_replace/2.

select_reverse(Cont)

-spec select_reverse(Cont) -> {[Match], Cont} | '$end_of_table'
                        when Cont :: continuation(), Match :: term().

Equivalent to ets:select_reverse/1.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:select_reverse/1.

select_reverse(Tab, MatchSpec)

Equivalent to select_reverse(Tab, MatchSpec, shards_meta:get(Tab)).

select_reverse(Tab, MatchSpec, LimitOrMeta)

-spec select_reverse(Tab, MatchSpec, LimitOrMeta) -> {[Match], Cont} | '$end_of_table' | [Match]
                        when
                            Tab :: tab(),
                            MatchSpec :: ets:match_spec(),
                            LimitOrMeta :: pos_integer() | shards_meta:t(),
                            Match :: term(),
                            Cont :: continuation().

If 3rd argument is pos_integer() this function behaves like ets:select_reverse/3, otherwise, the 3rd argument is assumed as shards_meta:t()` and it behaves like `ets:select_reverse/2.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:select_reverse/3.

select_reverse(Tab, MatchSpec, Limit, Meta)

-spec select_reverse(Tab, MatchSpec, Limit, Meta) -> {[Match], Cont} | '$end_of_table'
                        when
                            Tab :: tab(),
                            MatchSpec :: ets:match_spec(),
                            Limit :: pos_integer(),
                            Meta :: shards_meta:t(),
                            Match :: term(),
                            Cont :: continuation().

Equivalent to ets:select_reverse/3.

The order in which results are returned might be not the same as the original ETS function.

See also: ets:select_reverse/3.

setopts(Tab, Opts)

Equivalent to setopts(Tab, Opts, shards_meta:get(Tab)).

setopts(Tab, Opts, Meta)

-spec setopts(Tab, Opts, Meta) -> true
                 when
                     Tab :: tab(),
                     Opts :: Opt | [Opt],
                     Opt :: {heir, pid(), HeirData} | {heir, none},
                     Meta :: shards_meta:t(),
                     HeirData :: term().

Equivalent to ets:setopts/2.

Returns true if the function was applied successfully on each partition, otherwise, false is returned.

See also: ets:setopts/2.

tab2file(Tab, Filename)

Equivalent to tab2file(Tab, Filename, []).

tab2file(Tab, Filename, Options)

-spec tab2file(Tab, Filename, Options) -> ok | {error, Reason}
                  when
                      Tab :: tab(),
                      Filename :: filename(),
                      Options :: [Option],
                      Option :: {extended_info, [md5sum | object_count]} | {sync, boolean()},
                      Reason :: term().

Equivalent to ets:tab2file/3.

This function generates one file per partition using ets:tab2file/3, and also generates a master file with the given Filename that holds the information of the created partition files so that they can be recovered by calling ets:file2tab/1,2.

See also: ets:tab2file/3.

tab2list(Tab)

Equivalent to tab2list(Tab, shards_meta:get(Tab)).

tab2list(Tab, Meta)

-spec tab2list(Tab, Meta) -> [Object] when Tab :: tab(), Meta :: shards_meta:t(), Object :: tuple().

Equivalent to ets:tab2list/1.

See also: ets:tab2list/1.

tabfile_info(Filename)

-spec tabfile_info(Filename) -> {ok, TableInfo} | {error, Reason}
                      when Filename :: filename(), TableInfo :: [tabinfo_item()], Reason :: term().

Equivalent to ets:tabfile_info/1.

Adds extra information about the partitions.

See also: ets:tabfile_info/1.

table(Tab)

Equivalent to table(Tab, []).

table(Tab, Options)

Equivalent to table(Tab, Options, shards_meta:get(Tab)).

table(Tab, Options, Meta)

-spec table(Tab, Options, Meta) -> QueryHandle
               when
                   Tab :: tab(),
                   QueryHandle :: qlc:query_handle(),
                   Options :: [Option] | Option,
                   Meta :: shards_meta:t(),
                   Option :: {n_objects, NObjects} | {traverse, TraverseMethod},
                   NObjects :: default | pos_integer(),
                   MatchSpec :: ets:match_spec(),
                   TraverseMethod :: first_next | last_prev | select | {select, MatchSpec}.

Similar to ets:table/2, but it returns a list of qlc:query_handle(); one per partition.

See also: ets:table/2.

table_meta(Tab)

-spec table_meta(Tab :: tab()) -> shards_meta:t().

Returns the metadata associated with the given table Tab.

take(Tab, Key)

Equivalent to take(Tab, Key, shards_meta:get(Tab)).

take(Tab, Key, Meta)

-spec take(Tab, Key, Meta) -> [Object]
              when Tab :: tab(), Key :: term(), Meta :: shards_meta:t(), Object :: tuple().

Equivalent to ets:take/2.

See also: ets:take/2.

test_ms(Tuple, MatchSpec)

Equivalent to ets:test_ms(Tuple, MatchSpec).

update_counter(Tab, Key, UpdateOp)

Equivalent to update_counter(Tab, Key, UpdateOp, shards_meta:get(Tab)).

update_counter(Tab, Key, UpdateOp, DefaultOrMeta)

-spec update_counter(Tab, Key, UpdateOp, DefaultOrMeta) -> Result | [Result]
                        when
                            Tab :: tab(),
                            Key :: term(),
                            UpdateOp :: term(),
                            DefaultOrMeta :: tuple() | shards_meta:t(),
                            Result :: integer().

Equivalent to ets:update_counter/4.

If the 4th argument is shards_meta:t(), it behaves like ets:update_counter/3.

See also: ets:update_counter/4.

update_counter(Tab, Key, UpdateOp, Default, Meta)

-spec update_counter(Tab, Key, UpdateOp, Default, Meta) -> Result | [Result]
                        when
                            Tab :: tab(),
                            Key :: term(),
                            UpdateOp :: term(),
                            Default :: tuple(),
                            Meta :: shards_meta:t(),
                            Result :: integer().

Equivalent to ets:update_counter/4.

See also: ets:update_counter/4.

update_element(Tab, Key, ElementSpec)

Equivalent to update_element(Tab, Key, ElementSpec, shards_meta:get(Tab)).

update_element(Tab, Key, ElementSpec, Meta)

-spec update_element(Tab, Key, ElementSpec, Meta) -> boolean()
                        when
                            Tab :: tab(),
                            Key :: term(),
                            ElementSpec :: {Pos, Value} | [{Pos, Value}],
                            Pos :: pos_integer(),
                            Value :: term(),
                            Meta :: shards_meta:t().

Equivalent to ets:update_element/3.

See also: ets:update_element/3.