Pockets (Pockets v1.0.0) View Source

Pockets is a wrapper around Erlang :ets and :dets, built-in options for memory- and disk-based term storage. It offers simple key/value storage using an interface similar to the Map or Keyword modules. This can be a useful persistent cache for many use cases.

Note that this package and the libraries that underpin it may have limitations or specific behaviors that may affect its suitability for various use-cases. For example, the limited support for concurrency provided by the :ets(3) module is not yet provided by :dets.

See also

Link to this section Summary

Types

A table alias is used to refer to a Pockets table: it is usually an atom, but in some cases it may be a reference (e.g. in an :ets table with named_table: false).

Functions

Closes the given table (i.e. the opposite of open/3). Only processes that have opened a table are allowed to close it. For :ets in-memory tables, this is the same as destroy/1. For :dets disk-based tables, this will save any outstanding changes to the file.

Deletes the entry in table for a specific key. Because this returns the table alias, you can pipe multiple operations together.

Destroys the given table. For disk-based (:dets) tables, this will delete the backing file. For memory-based (:ets) tables, this destroys the table and its contents.

Checks if the given table is empty.

Gets the value for a specific key in the table.

Checks if the table has the given key.

Gets info about the given table.

Gets info about the given item in the table. The available items depend on the type of table.

Gets a list of keys in the given table. For larger tables, consider keys_stream/1

Gets a list of keys in the given table as a stream.

This is a powerful function that lets you merge input into an open table. All data in the input will be added to the table: the keys in the input "have precedence" over pre-existing keys in the table.

Creates a new table either in memory (default) or on disk.

Open a table for use. The behavior and available options of this function vary depending on the storage mechanism of the table (i.e.memory or disk).

Puts the given value under key in given table.

Both :ets and :dets files can be saved to disk. You can use this function to persist an in-memory :ets file to disk for later use, or you can use it to make a copy of an existing :dets table.

Show all registered Pockets tables.

Returns the size of the given table, measured by the number of entries.

Outputs the contents of the given table to a list.

Outputs the contents of the table to a map.

Outputs the contents of the table to a stream for lazy evaluation.

Truncates the given table; this removes all entries from the table while leaving its options intact.

Link to this section Types

Specs

alias() :: atom() | reference()

A table alias is used to refer to a Pockets table: it is usually an atom, but in some cases it may be a reference (e.g. in an :ets table with named_table: false).

Link to this section Functions

Specs

close(table_alias :: alias()) :: :ok | {:error, any()}

Closes the given table (i.e. the opposite of open/3). Only processes that have opened a table are allowed to close it. For :ets in-memory tables, this is the same as destroy/1. For :dets disk-based tables, this will save any outstanding changes to the file.

Link to this function

delete(table_alias, key)

View Source

Specs

delete(table_alias :: alias(), any()) :: alias() | {:error, any()}

Deletes the entry in table for a specific key. Because this returns the table alias, you can pipe multiple operations together.

Examples

iex> Pockets.new(:my_cache)
{:ok, :my_cache}
iex> Pockets.merge(:my_cache, %{a: "apple", b: "boy", c: "cat"})
:my_cache
iex> Pockets.to_map(:my_cache)
%{a: "apple", b: "boy", c: "cat"}
iex> Pockets.delete(:my_cache, :b)
:my_cache
iex> Pockets.to_map(:my_cache)
%{a: "apple", c: "cat"}

Specs

destroy(table_alias :: alias()) :: :ok | {:error, any()}

Destroys the given table. For disk-based (:dets) tables, this will delete the backing file. For memory-based (:ets) tables, this destroys the table and its contents.

Examples

iex> Pockets.new(:my_cache, "/tmp/cache.dets")
{:ok, :my_cache}
iex> Pockets.destroy(:my_cache)
:ok

Specs

empty?(table_alias :: alias()) :: boolean()

Checks if the given table is empty.

Examples

iex> {:ok, tid} = Pockets.new(:my_cache)
{:ok, :my_cache}
iex> Pockets.empty?(tid)
true
Link to this function

get(table_alias, key, default \\ nil)

View Source

Specs

get(table_alias :: alias(), any(), any()) :: alias()

Gets the value for a specific key in the table.

For tables of type :set, this function behaves like Map.get/3: the value is returned (if present), otherwise the default is returned.

However, for tables of type :bag and :duplicate_bag, this function will always return a list and the default is ignored. This has to do with keyword lists being the underlying data structure powering all of this.

Examples

iex> Pockets.new(:t1, :memory, type: :set)
{:ok, :t1}
iex> Pockets.get(:t1, :x, "Default Value")
"Default Value"

iex> Pockets.new(:t2, :memory, type: :bag)
{:ok, :t2}
iex> Pockets.get(:t2, :x, "Ignored Default Value")
[]

# Bag
iex> Pockets.new(:t_bag, :memory, type: :bag)
{:ok, :t_bag}
iex> Pockets.put(:t_bag, :c, "Cream")
:t_bag
iex> Pockets.put(:t_bag, :c, "Sugar")
:t_bag
iex> Pockets.put(:t_bag, :c, "Sugar")
:t_bag
iex> Pockets.get(:t_bag, :c)
["Cream", "Sugar"]

# Duplicate Bag
iex> Pockets.new(:t_dupe_bag, :memory, type: :duplicate_bag)
{:ok, :t_dupe_bag}
iex> Pockets.put(:t_dupe_bag, :c, "Cream")
:t_bag
iex> Pockets.put(:t_dupe_bag, :c, "Sugar")
:t_bag
iex> Pockets.put(:t_dupe_bag, :c, "Sugar")
:t_bag
iex> Pockets.get(:t_bag, :c)
["Cream", "Sugar", "Sugar"]
Link to this function

has_key?(table_alias, key)

View Source

Specs

has_key?(table_alias :: alias(), any()) :: boolean()

Checks if the table has the given key.

Specs

info(table_alias :: alias()) :: Pockets.EtsInfo.t() | Pockets.DetsInfo.t()

Gets info about the given table.

Specs

info(table_alias :: alias(), atom()) :: any()

Gets info about the given item in the table. The available items depend on the type of table.

Link to this macro

is_alias(value)

View Source (macro)

Specs

keys(table_alias :: alias()) :: list()

Gets a list of keys in the given table. For larger tables, consider keys_stream/1

Link to this function

keys_stream(table_alias)

View Source

Gets a list of keys in the given table as a stream.

Link to this function

merge(table_alias, input)

View Source

Specs

merge(alias(), input :: alias() | list() | map()) :: alias()

This is a powerful function that lets you merge input into an open table. All data in the input will be added to the table: the keys in the input "have precedence" over pre-existing keys in the table.

When the input to be merged is...

  • an alias for another table, the contents from that table are added to the given table_alias
  • a map, the contents from the map are added into the given table_alias
  • a list, the contents from the list are added into the given table_alias

Examples

# Merging a map into a table:
iex> Pockets.new(:my_cache)
{:ok, :my_cache}
iex> Pockets.merge(:my_cache, %{a: "apple", b: "boy", c: "cat"})
:my_cache
iex> Pockets.to_map(:my_cache)
%{a: "apple", b: "boy", c: "cat"}

# Merging two tables:
iex> Pockets.new(:my_first)
{:ok, :my_first}
iex> Pockets.merge(:my_first, %{a: "apple", b: "boy", c: "cat"})
:my_first
iex> Pockets.new(:my_second)
{:ok, :my_second}
iex> Pockets.merge(:my_second, %{x: "xray", y: "yellow", z: "zebra"})
:my_second
iex> Pockets.merge(:my_first, :my_second)
:my_first
iex> Pockets.to_map(:my_first)
%{a: "apple", b: "boy", c: "cat", x: "xray", y: "yellow", z: "zebra"}
Link to this function

new(table_alias, storage \\ :memory, opts \\ [type: :set])

View Source

Specs

new(table_alias :: alias(), :memory | binary(), opts :: keyword()) ::
  {:ok, alias()} | {:error, any()}

Creates a new table either in memory (default) or on disk.

The second argument specifies the storage mechanism for the table, either a path to a file (as a string) for disk-backed tables (:dets), or in :memory for memory-backed tables (:ets).

The available opts are specific to the storage engine being used.

In Memory

When creating a new memory-based table (i.e. the second argument is :memory or is omitted), the following options are supported:

  • :type One of [:bag, :duplicate_bag, :set] Default: set
  • :access One of :public | :protected | :private. Default: :public
  • :named_table boolean affects how the table is referenced. If false, the table alias will be a reference (not an atom). Default: true
  • :keypos integer indicating the position of the element of each object to be used as key. Default: 1
  • :read_concurrency boolean. Default: true
  • :write_concurrency boolean. Default: true
  • :decentralized_counters boolean. Default: false
  • :compressed boolean. Default: false

The full list of default options for memory-based tables are:

[type: :set, access: :public, named_table: true, keypos: 1, read_concurrency: true, write_concurrency: true, decentralized_counters: false, compressed: false]

The structure of the options is changed from the original to make them more compatible with Elixir conventions. See the original documentation for specifics about each option.

File-based Tables

When creating a new disk-based table (i.e. the second argument is a string path to a file), the following options are supported:

  • :type One of [:bag, :duplicate_bag, :set] Default: set
  • :access One of :read | :read_write. Default: :read_write
  • :auto_save integer or :infinity specifying the autosave interval (where :infinity disables the feature). Default: 180000 (3 minutes)
  • :keypos integer indicating the position of the element of each object to be used as key. Default: 1
  • :file The name of the file to be opened. Defaults to the table name. (Use of this confusing option is not recommended)
  • :max_no_slots The maximum number of slots to be used. Default: 32 M (million?)
  • :min_no_slots Application performance can be enhanced with this flag by specifying the estimated number of different keys to be stored in the table. Default: 256 (minimum)
  • :ram_file boolean. Whether the table is to be kept in RAM. Keeping the table in RAM can sound like an anomaly, but can enhance the performance of applications that open a table, insert a set of objects, and then close the table. When the table is closed, its contents are written to the disk file. Default: false
  • :repair boolean or :force. The flag specifies if the :dets server invokes the automatic file reparation algorithm. Default: true

The default options for disk-based tables are:

[type: :set]

Examples

iex> Pockets.new(:ram_cache, :memory)
{:ok, :ram_cache}

iex> Pockets.new(:disk_cache, "/tmp/my.dets")
{:ok, :disk_cache}
Link to this function

open(table_alias, storage \\ :memory, opts \\ [type: :set])

View Source

Specs

open(alias(), :memory | binary(), opts :: keyword()) :: any()

Open a table for use. The behavior and available options of this function vary depending on the storage mechanism of the table (i.e.memory or disk).

Memory-based Tables

For memory-based tables open/3 is synonymous with new/3.

Disk-based Tables

Because a file is involved, the :create? option provides a bit more control over whether or not

Options

  • :create? Indicates whether a new table should be created if the one being requested does not already exist. Default: false

If :create? is true and the filepath indicated by the second argument does not exist, open/3 is synonymous with new/3.

Examples

iex> Pockets.open(:my_cache)
{:ok, :my_cache}
iex> Pockets.open(:my_cache, :memory)
{:ok, :my_cache}
iex> Pockets.open(:disk_cache, "/tmp/cache.dets")
{:ok, :disk_cache}
Link to this function

put(table_alias, key, value)

View Source

Specs

put(table_alias :: alias(), any(), any()) :: alias()

Puts the given value under key in given table.

For tables of type :set, this will behave like Map.put/3.

Other table types aren't as straightforward. See the examples under get/3

Examples

# Set
iex> Pockets.new(:t_set)
{:ok, :t_set}
iex> Pockets.put(:t_set, :z, "Zulu")
:t_set
iex> Pockets.get(:t_set, :z)
"Zulu"
Link to this function

save_as(table_alias, target_file, opts \\ [])

View Source

Specs

save_as(table_alias :: alias(), binary(), keyword()) :: :ok | {:error, any()}

Both :ets and :dets files can be saved to disk. You can use this function to persist an in-memory :ets file to disk for later use, or you can use it to make a copy of an existing :dets table.

The target file must not be in use by another table; if the target file exists this will return an error unless the :overwrite? option is set to true.

Options:

  • overwrite? default: false

Show all registered Pockets tables.

Examples

iex> Pockets.show_tables()
[%Pockets.Table{alias: :my_cache, library: :ets, tid: :my_cache, type: :set}]

Specs

size(table_alias :: alias()) :: integer()

Returns the size of the given table, measured by the number of entries.

Specs

to_list(table_alias :: alias()) :: list()

Outputs the contents of the given table to a list.

Although this is useful for debugging purposes, for larger data sets consider using to_stream/1 instead.

Specs

to_map(table_alias :: alias()) :: map()

Outputs the contents of the table to a map.

Although this is useful for debugging purposes, for larger data sets consider using to_stream/1 instead.

Outputs the contents of the table to a stream for lazy evaluation.

Specs

truncate(table_alias :: alias()) :: alias() | {:error, any()}

Truncates the given table; this removes all entries from the table while leaving its options intact.

Examples

iex> Pockets.put(:my_cache, :a, "Apple") |> Pockets.put(:b, "boy") |> Pockets.put(:c, "Charlie")
:my_cache
iex> Pockets.truncate(:my_cache)
:my_cache
iex> Pockets.to_map(:my_cache)
%{}