Goblin (Goblin v0.6.0)

View Source

A lightweight, embedded database for Elixir.

Goblin is a persistent key-value store with ACID transactions, crash recovery, and automatic background compaction. It runs inside your application's supervision tree.

Starting a database

{:ok, db} = Goblin.start_link(
  name: MyApp.DB,
  data_dir: "/path/to/db"
)

Basic operations

Goblin.put(db, :alice, "Alice")
Goblin.get(db, :alice)
# => "Alice"

Goblin.remove(db, :alice)
Goblin.get(db, :alice)
# => nil

Batch operations

Goblin.put_multi(db, [{:alice, "Alice"}, {:bob, "Bob"}])

Goblin.get_multi(db, [:alice, :bob])
# => [{:alice, "Alice"}, {:bob, "Bob"}]

Transactions

Goblin.transaction(db, fn tx ->
  counter = Goblin.Tx.get(tx, :counter, default: 0)
  tx = Goblin.Tx.put(tx, :counter, counter + 1)
  {:commit, tx, :ok}
end)
# => :ok

See start_link/1 for configuration options.

Summary

Functions

Returns a specification to start this module under a supervisor.

Returns whether background compaction is currently running.

Exports a snapshot of the database as a .tar.gz archive.

Returns whether a memory-to-disk flush is currently running.

Retrieves the value associated with a key.

Retrieves values for multiple keys in a single read.

Writes a key-value pair to the database.

Writes multiple key-value pairs in a single transaction.

Performs a read transaction.

Removes a key from the database.

Removes multiple keys from the database in a single transaction.

Returns a stream of key-value pairs, optionally bounded by a range.

Starts the database.

Subscribes the calling process to write notifications.

Executes a function within a write transaction.

Unsubscribes the calling process from write notifications.

Types

db_key()

@type db_key() :: term()

db_tag()

@type db_tag() :: term()

db_value()

@type db_value() :: term()

Functions

child_name(namespace, suffix)

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

compacting?(db)

@spec compacting?(Supervisor.supervisor()) :: boolean()

Returns whether background compaction is currently running.

export(db, export_dir)

@spec export(Supervisor.supervisor(), Path.t()) :: {:ok, Path.t()} | {:error, term()}

Exports a snapshot of the database as a .tar.gz archive.

The archive can be unpacked and used as the data_dir for a new database instance, acting as a backup.

The export runs inside a read transaction, preventing file deletion while the snapshot is being created.

Parameters

  • db - The database server (PID or registered name)
  • export_dir - Directory to place the exported .tar.gz file

Returns

  • {:ok, export_path} - Path to the created archive
  • {:error, reason} - If an error occurred

Examples

Goblin.export(db, "/backups")
# => {:ok, "/backups/goblin_20260220T120000Z.tar.gz"}

flushing?(db)

@spec flushing?(Supervisor.supervisor()) :: boolean()

Returns whether a memory-to-disk flush is currently running.

get(db, key, opts \\ [])

@spec get(Supervisor.supervisor(), db_key(), keyword()) :: db_value() | nil

Retrieves the value associated with a key.

Returns the default value if the key is not found.

Parameters

  • db - The database server (PID or registered name)
  • key - The key to look up
  • opts - A keyword list with the following options (default: []):
    • :tag - Tag the key is namespaced under
    • :default - Value to return if key is not found (default: nil)

Returns

  • The value associated with the key, or default if not found

Examples

Goblin.get(db, :alice)
# => "Alice"

Goblin.get(db, :nonexistent)
# => nil

Goblin.get(db, :nonexistent, default: :not_found)
# => :not_found

Goblin.get(db, :alice, tag: :admins)
# => "Alice"

get_multi(db, keys, opts \\ [])

@spec get_multi(Supervisor.supervisor(), [db_key()], keyword()) :: [
  {db_key(), db_value()}
]

Retrieves values for multiple keys in a single read.

Keys not found in the database are excluded from the result.

Parameters

  • db - The database server (PID or registered name)
  • keys - A list of keys to look up
  • opts - A keyword list with the following options (default: []):
    • :tag - Tag the keys are namespaced under

Returns

  • A list of {key, value} tuples for keys found, sorted by key

Examples

Goblin.get_multi(db, [:alice, :bob])
# => [{:alice, "Alice"}, {:bob, "Bob"}]

Goblin.get_multi(db, [:alice, :nonexistent])
# => [{:alice, "Alice"}]

put(db, key, value, opts \\ [])

Writes a key-value pair to the database.

Parameters

  • db - The database server (PID or registered name)
  • key - Any Elixir term to use as the key
  • value - Any Elixir term to associate with key
  • opts - A keyword list with the following options (default: []):
    • :tag - Tag to namespace the key under

Returns

  • :ok

Examples

Goblin.put(db, :alice, "Alice")
# => :ok

Goblin.put(db, :alice, "Alice", tag: :admins)
# => :ok

put_multi(db, pairs, opts \\ [])

@spec put_multi(Supervisor.supervisor(), [{db_key(), db_value()}], keyword()) :: :ok

Writes multiple key-value pairs in a single transaction.

Parameters

  • db - The database server (PID or registered name)
  • pairs - A list of {key, value} tuples
  • opts - A keyword list with the following options (default: []):
    • :tag - Tag to namespace the keys under

Returns

  • :ok

Examples

Goblin.put_multi(db, [{:alice, "Alice"}, {:bob, "Bob"}, {:charlie, "Charlie"}])
# => :ok

read(db, f)

@spec read(Supervisor.supervisor(), (Goblin.Tx.t() -> any())) :: any()

Performs a read transaction.

A snapshot is taken to provide a consistent view of the database. Multiple readers run concurrently without blocking each other. Attempting to write within a read transaction raises.

Parameters

  • db - The database server (PID or registered name)
  • f - A function that takes a Goblin.Tx.t() struct

Returns

  • The return value of f

Examples

Goblin.read(db, fn tx ->
  alice = Goblin.Tx.get(tx, :alice)
  bob = Goblin.Tx.get(tx, :bob)
  {alice, bob}
end)
# => {"Alice", "Bob"}

remove(db, key, opts \\ [])

@spec remove(Supervisor.supervisor(), db_key(), keyword()) :: :ok

Removes a key from the database.

Parameters

  • db - The database server (PID or registered name)
  • key - The key to remove
  • opts - A keyword list with the following options (default: []):
    • :tag - Tag the key is namespaced under

Returns

  • :ok

Examples

Goblin.remove(db, :alice)
# => :ok

Goblin.get(db, :alice)
# => nil

remove_multi(db, keys, opts \\ [])

@spec remove_multi(Supervisor.supervisor(), [db_key()], keyword()) :: :ok

Removes multiple keys from the database in a single transaction.

Parameters

  • db - The database server (PID or registered name)
  • keys - A list of keys to remove
  • opts - A keyword list with the following options (default: []):
    • :tag - Tag the keys are namespaced under

Returns

  • :ok

Examples

Goblin.remove_multi(db, [:alice, :bob, :charlie])
# => :ok

select(db, opts \\ [])

@spec select(
  Supervisor.supervisor(),
  keyword()
) :: Enumerable.t({db_key(), db_value()})

Returns a stream of key-value pairs, optionally bounded by a range.

Entries are sorted by key in ascending order. Both min and max are inclusive.

Parameters

  • db - The database server (PID or registered name)
  • opts - Keyword list of options:
    • :min - Minimum key, inclusive (optional)
    • :max - Maximum key, inclusive (optional)
    • :tag - Tag to filter by (optional)

Returns

  • A stream of {key, value} tuples

Examples

Goblin.select(db) |> Enum.to_list()
# => [{:alice, "Alice"}, {:bob, "Bob"}, {:charlie, "Charlie"}]

Goblin.select(db, min: :bob) |> Enum.to_list()
# => [{:bob, "Bob"}, {:charlie, "Charlie"}]

Goblin.select(db, min: :alice, max: :bob) |> Enum.to_list()
# => [{:alice, "Alice"}, {:bob, "Bob"}]

start_link(opts)

@spec start_link(keyword()) :: Supervisor.on_start()

Starts the database.

Creates the data_dir if it does not exist.

Options

  • :name - Registered name for the database (optional, defaults to Goblin)
  • :data_dir - Directory path for database files (required)
  • :mem_limit - Bytes to buffer in memory before flushing to disk (default: 64 MB)
  • :bf_fpp - Bloom filter false positive probability (default: 0.01)

Returns

  • {:ok, pid} - On successful start
  • {:error, reason} - On failure

Examples

{:ok, db} = Goblin.start_link(
  name: MyApp.DB,
  data_dir: "/var/lib/myapp/db"
)

stop(db, reason \\ :normal, timeout \\ :infinity)

@spec stop(Supervisor.supervisor(), term(), non_neg_integer() | :infinity) :: :ok

Stops the database.

subscribe(db)

@spec subscribe(Supervisor.supervisor()) :: :ok | {:error, term()}

Subscribes the calling process to write notifications.

After subscribing, the process receives {:put, key, value} or {:remove, key} messages for each committed write. Batch writes produce one message per key.

Parameters

  • db - The database server (PID or registered name)

Returns

  • :ok

Examples

Goblin.subscribe(db)
# => :ok

Goblin.put(db, :alice, "Alice")

receive do
  {:put, :alice, value} -> value
end
# => "Alice"

transaction(db, f)

@spec transaction(Supervisor.supervisor(), (Goblin.Tx.t() -> Goblin.Tx.return())) ::
  any() | {:error, :aborted | :already_in_tx | :not_writer}

Executes a function within a write transaction.

Transactions are executed serially and are ACID-compliant. The provided function receives a transaction struct and must return {:commit, tx, reply} to commit, or :abort to abort. Returning anything else raises.

Parameters

  • db - The database server (PID or registered name)
  • f - A function that takes a Goblin.Tx.t() and returns a transaction result

Returns

  • reply - The value from {:commit, tx, reply} when committed
  • {:error, :aborted} - When the transaction is aborted

Examples

Goblin.transaction(db, fn tx ->
  counter = Goblin.Tx.get(tx, :counter, default: 0)
  tx = Goblin.Tx.put(tx, :counter, counter + 1)
  {:commit, tx, :ok}
end)
# => :ok

Goblin.transaction(db, fn _tx ->
  :abort
end)
# => {:error, :aborted}

unsubscribe(db)

@spec unsubscribe(Supervisor.supervisor()) :: :ok | {:error, term()}

Unsubscribes the calling process from write notifications.