trove

An embedded, crash-safe key-value store for Gleam.

trove stores data in an append-only B+ tree on disk. Every write appends new nodes and creates a new root — old data is never overwritten. This gives you crash safety, zero-cost MVCC snapshots, and single-writer / multiple-reader concurrency backed by an OTP actor.

Quick Start

import gleam/string
import trove
import trove/codec

let config = trove.Config(
  path: "./my_db",
  key_codec: codec.string(),
  value_codec: codec.string(),
  key_compare: string.compare,
  auto_compact: trove.NoAutoCompact,
  auto_file_sync: trove.AutoSync,
  call_timeout: 5000,
)

let assert Ok(db) = trove.open(config)
trove.put(db, key: "language", value: "gleam")
let assert Ok("gleam") = trove.get(db, key: "language")
trove.close(db)

Types

Controls automatic compaction behavior. When auto-compaction is enabled, compaction triggers after a write if both min_dirt and min_dirt_factor thresholds are exceeded simultaneously.

Note: Auto-compaction runs synchronously inside the database actor. While compaction is in progress, all other operations (reads, writes, snapshots) are queued and may time out on large databases. For latency-sensitive workloads, prefer NoAutoCompact and call compact manually from a separate process with an appropriate timeout.

pub type AutoCompact {
  AutoCompact(min_dirt: Int, min_dirt_factor: Float)
  NoAutoCompact
}

Constructors

  • AutoCompact(min_dirt: Int, min_dirt_factor: Float)

    Enable auto-compaction. min_dirt is the minimum number of mutation operations (inserts, updates, and deletes each add one to the dirt count) and min_dirt_factor is the minimum dirt ratio (0.0–1.0) — both must be exceeded for compaction to trigger.

  • NoAutoCompact

    Disable auto-compaction. Compaction can still be triggered manually with compact.

Database configuration passed to open.

  • path — directory where store files are kept (created if needed)
  • key_codec / value_codec — how to serialize keys and values to bytes. Must satisfy decode(encode(v)) == Ok(v) and be deterministic (same input always produces the same bytes).
  • key_compare — total order over keys: must be deterministic, antisymmetric, and transitive. Must be consistent with key_codec — keys that compare Eq must encode to identical bytes.
  • auto_compact — controls automatic compaction after writes
  • auto_file_sync — controls whether writes are automatically fsynced
  • call_timeout — milliseconds to wait for actor responses (5000 is a good starting point)
pub type Config(k, v) {
  Config(
    path: String,
    key_codec: codec.Codec(k),
    value_codec: codec.Codec(v),
    key_compare: fn(k, k) -> order.Order,
    auto_compact: AutoCompact,
    auto_file_sync: FileSync,
    call_timeout: Int,
  )
}

Constructors

Db

opaque </>

An open database handle. Parameterized by key type k and value type v.

pub opaque type Db(k, v)

Controls whether writes are automatically fsynced to disk.

pub type FileSync {
  AutoSync
  ManualSync
}

Constructors

  • AutoSync

    Automatically fsync after every write for maximum durability.

  • ManualSync

    Do not fsync automatically. Use file_sync to flush manually.

Errors that can occur when opening a database.

pub type OpenError {
  DirectoryError(detail: String)
  StoreError(detail: String)
  LockError(detail: String)
  ActorStartError
}

Constructors

  • DirectoryError(detail: String)

    The database directory could not be created or accessed.

  • StoreError(detail: String)

    The store file could not be opened or its header could not be recovered.

  • LockError(detail: String)

    The database path is already open by another actor on this node.

  • ActorStartError

    The OTP actor failed to start.

A point-in-time snapshot handle for consistent reads.

pub type Snapshot(k, v) =
  @internal Snapshot(k, v)

The result of a transaction callback. Return Commit to apply the transaction’s writes, or Cancel to discard them.

pub type TransactionResult(k, v, a) {
  Commit(tx: Tx(k, v), result: a)
  Cancel(result: a)
}

Constructors

  • Commit(tx: Tx(k, v), result: a)

    Apply the transaction’s writes and return the result value.

  • Cancel(result: a)

    Discard the transaction’s writes and return the result value.

A transaction handle for reading and writing within a transaction.

pub type Tx(k, v) =
  @internal Tx(k, v)

Values

pub fn close(db: Db(k, v)) -> Nil

Close the database and release the file handle and path lock. Does not fsync — if using ManualSync, call file_sync before closing to ensure durability. The Db handle must not be used after calling this.

Panics if the store file handle cannot be closed.

trove.close(db)
pub fn compact(
  db: Db(k, v),
  timeout timeout: Int,
) -> Result(Nil, String)

Trigger a manual compaction. Rebuilds the store file keeping only live entries, resetting the dirt factor to zero. Returns Ok(Nil) on success or Error(reason) if compaction failed. On failure the database remains functional with the original store file.

The timeout is separate from call_timeout because compaction can take much longer than normal operations.

let assert Ok(Nil) = trove.compact(db, timeout: 60_000)
pub fn delete(db: Db(k, v), key key: k) -> Nil

Remove a key. No error if the key does not exist.

Panics on store I/O errors (e.g. disk full, file corruption).

trove.delete(db, key: "hello")
pub fn delete_multi(db: Db(k, v), keys keys: List(k)) -> Nil

Atomically delete multiple keys.

Panics on store I/O errors (e.g. disk full, file corruption).

trove.delete_multi(db, keys: ["a", "b"])
pub fn dirt_factor(db: Db(k, v)) -> Float

Returns the current dirt factor: a float between 0.0 and 1.0 that approximates how much of the store file is occupied by superseded data. Overwrites and deletes increment the dirt counter because they write new nodes that make old ones unreachable. New inserts do not increment dirt since they don’t supersede existing data. The formula is dirt / (1 + size + dirt) — the +1 ensures the result is always well-defined, even for an empty tree. The value approaches but never reaches 1.0. Higher values mean more wasted space that compaction would reclaim.

let df = trove.dirt_factor(db)
pub fn file_sync(db: Db(k, v)) -> Nil

Force an fsync of the store file to disk. Useful when auto_file_sync is set to ManualSync and you want to control when data is flushed.

Panics if the fsync system call fails.

let config = trove.Config(..config, auto_file_sync: trove.ManualSync)
let assert Ok(db) = trove.open(config)
trove.put(db, key: "hello", value: "world")
trove.file_sync(db)
pub fn get(db: Db(k, v), key key: k) -> Result(v, Nil)

Look up a key. Returns Ok(value) if found, Error(Nil) if the key does not exist.

Panics on store I/O or decode errors (e.g. file corruption).

let assert Ok("world") = trove.get(db, key: "hello")
pub fn has_key(db: Db(k, v), key key: k) -> Bool

Check whether a key exists in the database.

Panics on store I/O or decode errors (e.g. file corruption).

let assert True = trove.has_key(db, key: "hello")
pub fn is_empty(db: Db(k, v)) -> Bool

Returns True if the database contains no entries.

let empty = trove.is_empty(db)
pub fn open(config: Config(k, v)) -> Result(Db(k, v), OpenError)

Open a database at the configured path. Creates the directory if it does not exist. If a store file already exists, recovers the tree from the latest valid header.

let config = trove.Config(
  path: "./my_db",
  key_codec: codec.string(),
  value_codec: codec.string(),
  key_compare: string.compare,
  auto_compact: trove.NoAutoCompact,
  auto_file_sync: trove.AutoSync,
  call_timeout: 5000,
)
let assert Ok(db) = trove.open(config)
pub fn put(db: Db(k, v), key key: k, value value: v) -> Nil

Insert or update a key-value pair.

Panics on store I/O errors (e.g. disk full, file corruption).

trove.put(db, key: "hello", value: "world")
pub fn put_and_delete_multi(
  db: Db(k, v),
  puts puts: List(#(k, v)),
  deletes deletes: List(k),
) -> Nil

Atomically insert and delete entries in a single operation. Puts are applied first, then deletes, all under a single header write.

Panics on store I/O errors (e.g. disk full, file corruption).

trove.put_and_delete_multi(
  db,
  puts: [#("new_key", "value")],
  deletes: ["old_key"],
)
pub fn put_multi(
  db: Db(k, v),
  entries entries: List(#(k, v)),
) -> Nil

Atomically insert multiple key-value pairs. A single header write covers the entire batch.

Panics on store I/O errors (e.g. disk full, file corruption).

trove.put_multi(db, entries: [#("a", "1"), #("b", "2")])
pub fn range(
  db db: Db(k, v),
  min min: option.Option(range.Bound(k)),
  max max: option.Option(range.Bound(k)),
  direction direction: range.Direction,
) -> List(#(k, v))

Iterate over entries in the database within optional key bounds. Returns a List of key-value pairs.

For large result sets, use with_snapshot and snapshot_range instead to stream entries lazily without loading them all at once.

Panics if the snapshot file handle cannot be opened, or on store read/decode errors during iteration.

Use range.Inclusive(key) or range.Exclusive(key) for bounds, or option.None for unbounded. Use range.Forward or range.Reverse for direction.

import gleam/option.{Some}
import trove/range

let results =
  trove.range(
    db,
    min: Some(range.Inclusive("a")),
    max: Some(range.Exclusive("z")),
    direction: range.Forward,
  )
pub fn set_auto_compact(
  db: Db(k, v),
  setting setting: AutoCompact,
) -> Nil

Change the auto-compaction setting at runtime.

trove.set_auto_compact(db, trove.AutoCompact(min_dirt: 100, min_dirt_factor: 0.25))
pub fn size(db: Db(k, v)) -> Int

Returns the number of live entries in the database.

let count = trove.size(db)
pub fn snapshot_get(
  snapshot snapshot: Snapshot(k, v),
  key key: k,
) -> Result(v, Nil)

Look up a key in a snapshot. Returns Error(Nil) if the key does not exist.

Panics on store read or decode errors (e.g. file corruption).

trove.with_snapshot(db, fn(snap) {
  let assert Ok(value) = trove.snapshot_get(snapshot: snap, key: "my_key")
  value
})
pub fn snapshot_range(
  snapshot snapshot: Snapshot(k, v),
  min min: option.Option(range.Bound(k)),
  max max: option.Option(range.Bound(k)),
  direction direction: range.Direction,
) -> yielder.Yielder(#(k, v))

Iterate over entries in a snapshot within optional key bounds. Returns a lazy Yielder that streams entries from disk on demand, reading only one leaf node at a time.

The yielder holds a reference to the snapshot’s file handle, so it must be consumed before the snapshot is closed. For large ranges, prefer this over range to avoid loading all entries into memory.

Panics on store read or decode errors during iteration (e.g. file corruption).

Use range.Inclusive(key) or range.Exclusive(key) for bounds, or option.None for unbounded. Use range.Forward or range.Reverse for direction.

import gleam/option.{None, Some}
import gleam/yielder
import trove/range

let entries = trove.with_snapshot(db, fn(snap) {
  let y = trove.snapshot_range(
    snapshot: snap,
    min: Some(range.Inclusive("a")),
    max: None,
    direction: range.Forward,
  )
  yielder.to_list(y)
})
pub fn transaction(
  db: Db(k, v),
  timeout timeout: Int,
  callback callback: fn(Tx(k, v)) -> TransactionResult(k, v, a),
) -> a

Run an atomic transaction. The callback receives a Tx handle and must return Commit(tx:, result: value) to apply writes or Cancel(result: value) to discard. The transaction holds exclusive write access for its duration.

The timeout parameter (in milliseconds) controls how long the caller waits for the transaction to complete, including queue wait time and callback execution. Choose a value appropriate for your workload — queued operations or auto-compaction may delay the start, and a long-running callback consumes the remaining budget.

Important: The callback runs inside the database actor. Do not call any trove functions (such as get, put, compact, etc.) on the same Db handle from within the callback — this will deadlock the actor until the call timeout fires. Use the Tx handle (tx_get, tx_put, tx_delete) for all reads and writes inside the transaction.

Panics if the Commit variant contains a stale or replaced Tx handle (e.g. the original handle instead of the latest one returned by tx_put/tx_delete).

Non-escaping: The Tx handle is only valid inside the callback. Do not store it in a variable, send it to another process, or return it — using a Tx after the callback returns will panic or produce undefined behavior.

Timeout semantics: If the timeout fires while the callback is still executing, the caller panics but the actor continues running the callback to completion. This means writes may be durably committed even though the caller observes a timeout failure. Choose a timeout that accommodates your expected callback duration and any queued operations ahead of it.

let result = trove.transaction(db, timeout: 5000, callback: fn(tx) {
  let tx = trove.tx_put(tx, key: "key", value: "value")
  trove.Commit(tx:, result: "done")
})
pub fn tx_delete(tx tx: Tx(k, v), key key: k) -> Tx(k, v)

Delete a key within a transaction. Returns the updated Tx.

Panics on store I/O errors (e.g. disk full, file corruption).

trove.transaction(db, timeout: 5000, callback: fn(tx) {
  let tx = trove.tx_delete(tx, key: "old_key")
  trove.Commit(tx:, result: Nil)
})
pub fn tx_get(tx tx: Tx(k, v), key key: k) -> Result(v, Nil)

Read a key within a transaction. Sees writes made earlier in the same transaction. Returns Error(Nil) if the key does not exist.

Panics on store I/O or decode errors (e.g. file corruption).

trove.transaction(db, timeout: 5000, callback: fn(tx) {
  let assert Ok(current) = trove.tx_get(tx, key: "counter")
  let tx = trove.tx_put(tx, key: "counter", value: current <> "!")
  trove.Commit(tx:, result: Nil)
})
pub fn tx_has_key(tx tx: Tx(k, v), key key: k) -> Bool

Check whether a key exists within a transaction. Sees writes made earlier in the same transaction.

Panics on store I/O or decode errors (e.g. file corruption).

trove.transaction(db, timeout: 5000, callback: fn(tx) {
  let exists = trove.tx_has_key(tx, key: "counter")
  trove.Commit(tx:, result: exists)
})
pub fn tx_put(
  tx tx: Tx(k, v),
  key key: k,
  value value: v,
) -> Tx(k, v)

Write a key-value pair within a transaction. Returns the updated Tx.

Panics on store I/O errors (e.g. disk full, file corruption).

trove.transaction(db, timeout: 5000, callback: fn(tx) {
  let tx = trove.tx_put(tx, key: "greeting", value: "hello")
  trove.Commit(tx:, result: Nil)
})
pub fn with_snapshot(
  db: Db(k, v),
  callback callback: fn(Snapshot(k, v)) -> a,
) -> a

Run a callback with a point-in-time snapshot. The snapshot sees the state of the database at the moment it was acquired — subsequent writes are invisible to it.

Non-escaping: The Snapshot handle is only valid inside the callback. Do not store it in a variable, send it to another process, or return it — using a Snapshot after the callback returns will panic or produce undefined behavior because the underlying file handle is closed on exit.

Panics if the snapshot file handle cannot be opened.

let result = trove.with_snapshot(db, fn(snap) {
  trove.snapshot_get(snapshot: snap, key: "my_key")
})
// result: Result(String, Nil)
Search Document