View Source DetsPlus (DetsPlus v2.4.2)
DetsPlus persistent tuple/struct/map storage.
DetsPlus has a similiar API as dets
but without
the 2GB file storage limit. Writes are buffered in an
internal ETS table and synced every auto_save
period
to the persistent storage.
While sync()
or auto_save
are in progress the database
is still readable and writeable.
There is no commitlog so not synced writes are lost.
Lookups are possible by key and non-matches are accelerated
using a bloom filter. The persistent file concept follows
DJ Bernsteins CDB database format, but uses an Elixir
encoded header https://cr.yp.to/cdb.html. When syncing
a new CDB database file is created and replaces
the old CDB atomically file using File.rename!
so
database corruptions are not possible from incomplete updates.
Limits are:
- Total file size: 18_446 Petabyte
- Maximum entry size: 4 Gigabyte
- Maximum entry count: :infinity
Example:
{:ok, dets} = DetsPlus.open_file(:example)
DetsPlus.insert(dets, {1, 1, 1})
[{1, 1, 1}] = DetsPlus.lookup(dets, 1)
:ok = DetsPlus.close(dets)
Summary
Functions
Returns a specification to start this module under a supervisor.
Syncs pending writes to the persistent file and closes the table.
Returns the number of object in the table. This is an estimate and the same as info(dets, :size)
.
Deletes all objects with key Key from table Name.
Deletes all objects from a table in almost constant time.
Deletes all instances of a specified object from a table.
Returns information about table Name as a list of objects
Returns the information associated with item
for the table. The following items are allowed
Inserts one or more objects into the table. If there already exists an object with a key matching the key of some of the given objects, the old object will be replaced.
Inserts one or more objects into the table. If there already exists an object with a key matching the key of some of the given objects, the old object will be retained and false will be returned.
Returns a list of all objects with key Key stored in the table.
Same as member?/2
Works like lookup/2
, but does not return the objects. Returns true if one or more table elements has the key key
, otherwise false.
Opens an existing table or creates a new table. If no
file
argument is provided the table name will be used.
Dets registers a Process under the provided name which can
be used for calling alternatively to the pid.
Reducer function following the Enum
protocol.
Same as open_file
but for embedding in a supervision tree.
Opens an existing table or creates a new table. If no
file
argument is provided the table name will be used.
Dets registers a Process under the provided name which can
be used for calling alternatively to the pid.
Starts a sync of all changes to the disk. Same as sync/1
but doesn't block
Ensures that all updates made to table are written to disk. While the sync is running the
table can still be used for reads and writes, but writes issued after the sync/1
call
will not be part of the persistent file. These new changes will only be included in the
next sync call.
Returns the contents of the table as list. Same as calling Enum.to_list()
Types
Functions
Returns a specification to start this module under a supervisor.
See Supervisor
.
Syncs pending writes to the persistent file and closes the table.
Returns the number of object in the table. This is an estimate and the same as info(dets, :size)
.
Deletes all objects with key Key from table Name.
Deletes all objects from a table in almost constant time.
Deletes all instances of a specified object from a table.
Returns information about table Name as a list of objects:
{file_size, integer() >= 0}}
- The file size, in bytes.{filename, file:name()}
- The name of the file where objects are stored.{keypos, keypos()}
- The key position.{size, integer() >= 0}
- The number of objects estimated in the table.{type, type()}
- The table type.
@spec info( t(), :file_size | :header_size | :filename | :keypos | :size | :type | :creation_stats | :bloom_bytes | :hashtable_bytes ) :: any()
Returns the information associated with item
for the table. The following items are allowed:
{file_size, integer() >= 0}}
- The file size, in bytes.{header_size, integer() >= 0}}
- The size of erlang term encoded header.{bloom_bytes, integer() >= 0}}
- The size of the in-memory and on-disk bloom filter, in bytes.{hashtable_bytes, integer() >= 0}}
- The size of the on-disk lookup hashtable, in bytes.{filename, file:name()}
- The name of the file where objects are stored.{keypos, keypos()}
- The key position.{size, integer() >= 0}
- The number of objects estimated in the table.{type, type()}
- The table type.
Inserts one or more objects into the table. If there already exists an object with a key matching the key of some of the given objects, the old object will be replaced.
Inserts one or more objects into the table. If there already exists an object with a key matching the key of some of the given objects, the old object will be retained and false will be returned.
Returns a list of all objects with key Key stored in the table.
Example:
2> State.open_file(:abc)
{ok,:abc}
3> State.insert(:abc, {1,2,3})
ok
4> State.insert(:abc, {1,3,4})
ok
5> State.lookup(:abc, 1).
[{1,3,4}]
If the table type is set, the function returns either the empty list or a list with one object, as there cannot be more than one object with a given key. If the table type is bag or duplicate_bag, the function returns a list of arbitrary length.
Notice that the order of objects returned is unspecified. In particular, the order in which objects were inserted is not reflected.
Same as member?/2
Works like lookup/2
, but does not return the objects. Returns true if one or more table elements has the key key
, otherwise false.
Opens an existing table or creates a new table. If no
file
argument is provided the table name will be used.
Dets registers a Process under the provided name which can
be used for calling alternatively to the pid.
Arguments:
file
- An optional path + filename for the database file.auto_save
- The autosave interval. If the interval is an integer Time, the table is flushed to disk whenever it is not accessed for Time milliseconds. If the interval is the atom infinity, autosave is disabled. Defaults to180_000
(3 minutes).auto_save_memory
- The autosave threshold in memory. When the internal ETS table reaches a size bigger than this the table is flushed to disk. Defaults to1_000_000_000
(1 GB)page_cache_memory
- The amount of memory to use for file system caching. Defaults to1_000_000_000
(1 GB)keypos
- The position of the element of each object to be used as key. Defaults to 1. The ability to explicitly state the key position is most convenient when we want to store Erlang records in which the first position of the record is the name of the record type.compressed
- Indicates whether the terms stored on disk should be compressed. Possible values are [true, false, 0 - 9]. Defaults tofalse
.
Reducer function following the Enum
protocol.
Same as open_file
but for embedding in a supervision tree.
Opens an existing table or creates a new table. If no
file
argument is provided the table name will be used.
Dets registers a Process under the provided name which can
be used for calling alternatively to the pid.
Arguments:
name
- Registered name for the dets .file
- An optional path + filename for the database file.auto_save
- The autosave interval. If the interval is an integer Time, the table is flushed to disk whenever it is not accessed for Time milliseconds. If the interval is the atom infinity, autosave is disabled. Defaults to180_000
(3 minutes).auto_save_memory
- The autosave threshold in memory. When the internal ETS table reaches a size bigger than this the table is flushed to disk. Defaults to1_000_000_000
(1 GB)page_cache_memory
- The amount of memory to use for file system caching. Defaults to1_000_000_000
(1 GB)keypos
- The position of the element of each object to be used as key. Defaults to 1. The ability to explicitly state the key position is most convenient when we want to store Erlang records in which the first position of the record is the name of the record type.compressed
- Indicates whether the terms stored on disk should be compressed. Possible values are [true, false, 0 - 9]. Defaults tofalse
.
Example:
defmodule MyApp.Application do
use Application
def start(_type, _args) do
children = [
{DetsPlus, name: :example}
]
opts = [strategy: :one_for_one, name: MyApp.Supervisor]
Supervisor.start_link(children, opts)
end
end
Starts a sync of all changes to the disk. Same as sync/1
but doesn't block
Ensures that all updates made to table are written to disk. While the sync is running the
table can still be used for reads and writes, but writes issued after the sync/1
call
will not be part of the persistent file. These new changes will only be included in the
next sync call.
Returns the contents of the table as list. Same as calling Enum.to_list()