trove
An embedded, crash-safe key-value store for Gleam.
trove stores data in an append-only B+ tree on disk. Every write appends new nodes and creates a new root — old data is never overwritten. This gives you crash safety, zero-cost MVCC snapshots, and single-writer / multiple-reader concurrency backed by an OTP actor.
Quick Start
import gleam/string
import trove
import trove/codec
let config = trove.Config(
path: "./my_db",
key_codec: codec.string(),
value_codec: codec.string(),
key_compare: string.compare,
auto_compact: trove.NoAutoCompact,
auto_file_sync: trove.AutoSync,
call_timeout: 5000,
)
let assert Ok(db) = trove.open(config)
trove.put(db, key: "language", value: "gleam")
let assert Ok("gleam") = trove.get(db, key: "language")
trove.close(db)
Types
Controls automatic compaction behavior. When auto-compaction is enabled,
compaction triggers after a write if both min_dirt and min_dirt_factor
thresholds are exceeded simultaneously.
Note: Auto-compaction runs synchronously inside the database actor.
While compaction is in progress, all other operations (reads, writes,
snapshots) are queued and may time out on large databases. For
latency-sensitive workloads, prefer NoAutoCompact and call compact
manually from a separate process with an appropriate timeout.
pub type AutoCompact {
AutoCompact(min_dirt: Int, min_dirt_factor: Float)
NoAutoCompact
}
Constructors
-
AutoCompact(min_dirt: Int, min_dirt_factor: Float)Enable auto-compaction.
min_dirtis the minimum number of mutation operations (inserts, updates, and deletes each add one to the dirt count) andmin_dirt_factoris the minimum dirt ratio (0.0–1.0) — both must be exceeded for compaction to trigger. -
NoAutoCompactDisable auto-compaction. Compaction can still be triggered manually with
compact.
Database configuration passed to open.
path— directory where store files are kept (created if needed)key_codec/value_codec— how to serialize keys and values to bytes. Must satisfydecode(encode(v)) == Ok(v)and be deterministic (same input always produces the same bytes).key_compare— total order over keys: must be deterministic, antisymmetric, and transitive. Must be consistent withkey_codec— keys that compareEqmust encode to identical bytes.auto_compact— controls automatic compaction after writesauto_file_sync— controls whether writes are automatically fsyncedcall_timeout— milliseconds to wait for actor responses (5000 is a good starting point)
pub type Config(k, v) {
Config(
path: String,
key_codec: codec.Codec(k),
value_codec: codec.Codec(v),
key_compare: fn(k, k) -> order.Order,
auto_compact: AutoCompact,
auto_file_sync: FileSync,
call_timeout: Int,
)
}
Constructors
-
Config( path: String, key_codec: codec.Codec(k), value_codec: codec.Codec(v), key_compare: fn(k, k) -> order.Order, auto_compact: AutoCompact, auto_file_sync: FileSync, call_timeout: Int, )
An open database handle. Parameterized by key type k and value type v.
pub opaque type Db(k, v)
Controls whether writes are automatically fsynced to disk.
pub type FileSync {
AutoSync
ManualSync
}
Constructors
-
AutoSyncAutomatically fsync after every write for maximum durability.
-
ManualSyncDo not fsync automatically. Use
file_syncto flush manually.
Errors that can occur when opening a database.
pub type OpenError {
DirectoryError(detail: String)
StoreError(detail: String)
LockError(detail: String)
ActorStartError
}
Constructors
-
DirectoryError(detail: String)The database directory could not be created or accessed.
-
StoreError(detail: String)The store file could not be opened or its header could not be recovered.
-
LockError(detail: String)The database path is already open by another actor on this node.
-
ActorStartErrorThe OTP actor failed to start.
A point-in-time snapshot handle for consistent reads.
pub type Snapshot(k, v) =
@internal Snapshot(k, v)
The result of a transaction callback. Return Commit to apply the
transaction’s writes, or Cancel to discard them.
pub type TransactionResult(k, v, a) {
Commit(tx: Tx(k, v), result: a)
Cancel(result: a)
}
Constructors
-
Commit(tx: Tx(k, v), result: a)Apply the transaction’s writes and return the result value.
-
Cancel(result: a)Discard the transaction’s writes and return the result value.
Values
pub fn close(db: Db(k, v)) -> Nil
Close the database and release the file handle and path lock. Does not
fsync — if using ManualSync, call file_sync before closing to ensure
durability. The Db handle must not be used after calling this.
Panics if the store file handle cannot be closed.
trove.close(db)
pub fn compact(
db: Db(k, v),
timeout timeout: Int,
) -> Result(Nil, String)
Trigger a manual compaction. Rebuilds the store file keeping only live
entries, resetting the dirt factor to zero. Returns Ok(Nil) on success
or Error(reason) if compaction failed. On failure the database remains
functional with the original store file.
The timeout is separate from call_timeout because compaction can take
much longer than normal operations.
let assert Ok(Nil) = trove.compact(db, timeout: 60_000)
pub fn delete(db: Db(k, v), key key: k) -> Nil
Remove a key. No error if the key does not exist.
Panics on store I/O errors (e.g. disk full, file corruption).
trove.delete(db, key: "hello")
pub fn delete_multi(db: Db(k, v), keys keys: List(k)) -> Nil
Atomically delete multiple keys.
Panics on store I/O errors (e.g. disk full, file corruption).
trove.delete_multi(db, keys: ["a", "b"])
pub fn dirt_factor(db: Db(k, v)) -> Float
Returns the current dirt factor: a float between 0.0 and 1.0 that
approximates how much of the store file is occupied by superseded data.
Overwrites and deletes increment the dirt counter because they write new
nodes that make old ones unreachable. New inserts do not increment dirt
since they don’t supersede existing data. The formula is
dirt / (1 + size + dirt) — the +1 ensures the result is always
well-defined, even for an empty tree. The value approaches but never
reaches 1.0. Higher values mean more wasted space that compaction would
reclaim.
let df = trove.dirt_factor(db)
pub fn file_sync(db: Db(k, v)) -> Nil
Force an fsync of the store file to disk. Useful when auto_file_sync
is set to ManualSync and you want to control when data is flushed.
Panics if the fsync system call fails.
let config = trove.Config(..config, auto_file_sync: trove.ManualSync)
let assert Ok(db) = trove.open(config)
trove.put(db, key: "hello", value: "world")
trove.file_sync(db)
pub fn get(db: Db(k, v), key key: k) -> Result(v, Nil)
Look up a key. Returns Ok(value) if found, Error(Nil) if the
key does not exist.
Panics on store I/O or decode errors (e.g. file corruption).
let assert Ok("world") = trove.get(db, key: "hello")
pub fn has_key(db: Db(k, v), key key: k) -> Bool
Check whether a key exists in the database.
Panics on store I/O or decode errors (e.g. file corruption).
let assert True = trove.has_key(db, key: "hello")
pub fn is_empty(db: Db(k, v)) -> Bool
Returns True if the database contains no entries.
let empty = trove.is_empty(db)
pub fn open(config: Config(k, v)) -> Result(Db(k, v), OpenError)
Open a database at the configured path. Creates the directory if it does not exist. If a store file already exists, recovers the tree from the latest valid header.
let config = trove.Config(
path: "./my_db",
key_codec: codec.string(),
value_codec: codec.string(),
key_compare: string.compare,
auto_compact: trove.NoAutoCompact,
auto_file_sync: trove.AutoSync,
call_timeout: 5000,
)
let assert Ok(db) = trove.open(config)
pub fn put(db: Db(k, v), key key: k, value value: v) -> Nil
Insert or update a key-value pair.
Panics on store I/O errors (e.g. disk full, file corruption).
trove.put(db, key: "hello", value: "world")
pub fn put_and_delete_multi(
db: Db(k, v),
puts puts: List(#(k, v)),
deletes deletes: List(k),
) -> Nil
Atomically insert and delete entries in a single operation. Puts are applied first, then deletes, all under a single header write.
Panics on store I/O errors (e.g. disk full, file corruption).
trove.put_and_delete_multi(
db,
puts: [#("new_key", "value")],
deletes: ["old_key"],
)
pub fn put_multi(
db: Db(k, v),
entries entries: List(#(k, v)),
) -> Nil
Atomically insert multiple key-value pairs. A single header write covers the entire batch.
Panics on store I/O errors (e.g. disk full, file corruption).
trove.put_multi(db, entries: [#("a", "1"), #("b", "2")])
pub fn range(
db db: Db(k, v),
min min: option.Option(range.Bound(k)),
max max: option.Option(range.Bound(k)),
direction direction: range.Direction,
) -> List(#(k, v))
Iterate over entries in the database within optional key bounds.
Returns a List of key-value pairs.
For large result sets, use with_snapshot and snapshot_range instead
to stream entries lazily without loading them all at once.
Panics if the snapshot file handle cannot be opened, or on store read/decode errors during iteration.
Use range.Inclusive(key) or range.Exclusive(key) for bounds,
or option.None for unbounded. Use range.Forward or range.Reverse
for direction.
import gleam/option.{Some}
import trove/range
let results =
trove.range(
db,
min: Some(range.Inclusive("a")),
max: Some(range.Exclusive("z")),
direction: range.Forward,
)
pub fn set_auto_compact(
db: Db(k, v),
setting setting: AutoCompact,
) -> Nil
Change the auto-compaction setting at runtime.
trove.set_auto_compact(db, trove.AutoCompact(min_dirt: 100, min_dirt_factor: 0.25))
pub fn size(db: Db(k, v)) -> Int
Returns the number of live entries in the database.
let count = trove.size(db)
pub fn snapshot_get(
snapshot snapshot: Snapshot(k, v),
key key: k,
) -> Result(v, Nil)
Look up a key in a snapshot. Returns Error(Nil) if the key does
not exist.
Panics on store read or decode errors (e.g. file corruption).
trove.with_snapshot(db, fn(snap) {
let assert Ok(value) = trove.snapshot_get(snapshot: snap, key: "my_key")
value
})
pub fn snapshot_range(
snapshot snapshot: Snapshot(k, v),
min min: option.Option(range.Bound(k)),
max max: option.Option(range.Bound(k)),
direction direction: range.Direction,
) -> yielder.Yielder(#(k, v))
Iterate over entries in a snapshot within optional key bounds.
Returns a lazy Yielder that streams entries from disk on demand,
reading only one leaf node at a time.
The yielder holds a reference to the snapshot’s file handle, so it
must be consumed before the snapshot is closed. For large ranges,
prefer this over range to avoid loading all entries into memory.
Panics on store read or decode errors during iteration (e.g. file corruption).
Use range.Inclusive(key) or range.Exclusive(key) for bounds,
or option.None for unbounded. Use range.Forward or range.Reverse
for direction.
import gleam/option.{None, Some}
import gleam/yielder
import trove/range
let entries = trove.with_snapshot(db, fn(snap) {
let y = trove.snapshot_range(
snapshot: snap,
min: Some(range.Inclusive("a")),
max: None,
direction: range.Forward,
)
yielder.to_list(y)
})
pub fn transaction(
db: Db(k, v),
timeout timeout: Int,
callback callback: fn(Tx(k, v)) -> TransactionResult(k, v, a),
) -> a
Run an atomic transaction. The callback receives a Tx handle and must
return Commit(tx:, result: value) to apply writes or
Cancel(result: value) to discard. The transaction holds exclusive
write access for its duration.
The timeout parameter (in milliseconds) controls how long the caller
waits for the transaction to complete, including queue wait time and
callback execution. Choose a value appropriate for your workload —
queued operations or auto-compaction may delay the start, and a
long-running callback consumes the remaining budget.
Important: The callback runs inside the database actor. Do not call
any trove functions (such as get, put, compact, etc.) on the
same Db handle from within the callback — this will deadlock the actor
until the call timeout fires. Use the Tx handle (tx_get, tx_put,
tx_delete) for all reads and writes inside the transaction.
Panics if the Commit variant contains a stale or replaced Tx
handle (e.g. the original handle instead of the latest one returned by
tx_put/tx_delete).
Non-escaping: The Tx handle is only valid inside the callback.
Do not store it in a variable, send it to another process, or return it —
using a Tx after the callback returns will panic or produce undefined
behavior.
Timeout semantics: If the timeout fires while the callback is still executing, the caller panics but the actor continues running the callback to completion. This means writes may be durably committed even though the caller observes a timeout failure. Choose a timeout that accommodates your expected callback duration and any queued operations ahead of it.
let result = trove.transaction(db, timeout: 5000, callback: fn(tx) {
let tx = trove.tx_put(tx, key: "key", value: "value")
trove.Commit(tx:, result: "done")
})
pub fn tx_delete(tx tx: Tx(k, v), key key: k) -> Tx(k, v)
Delete a key within a transaction. Returns the updated Tx.
Panics on store I/O errors (e.g. disk full, file corruption).
trove.transaction(db, timeout: 5000, callback: fn(tx) {
let tx = trove.tx_delete(tx, key: "old_key")
trove.Commit(tx:, result: Nil)
})
pub fn tx_get(tx tx: Tx(k, v), key key: k) -> Result(v, Nil)
Read a key within a transaction. Sees writes made earlier in the same
transaction. Returns Error(Nil) if the key does not exist.
Panics on store I/O or decode errors (e.g. file corruption).
trove.transaction(db, timeout: 5000, callback: fn(tx) {
let assert Ok(current) = trove.tx_get(tx, key: "counter")
let tx = trove.tx_put(tx, key: "counter", value: current <> "!")
trove.Commit(tx:, result: Nil)
})
pub fn tx_has_key(tx tx: Tx(k, v), key key: k) -> Bool
Check whether a key exists within a transaction. Sees writes made earlier in the same transaction.
Panics on store I/O or decode errors (e.g. file corruption).
trove.transaction(db, timeout: 5000, callback: fn(tx) {
let exists = trove.tx_has_key(tx, key: "counter")
trove.Commit(tx:, result: exists)
})
pub fn tx_put(
tx tx: Tx(k, v),
key key: k,
value value: v,
) -> Tx(k, v)
Write a key-value pair within a transaction. Returns the updated Tx.
Panics on store I/O errors (e.g. disk full, file corruption).
trove.transaction(db, timeout: 5000, callback: fn(tx) {
let tx = trove.tx_put(tx, key: "greeting", value: "hello")
trove.Commit(tx:, result: Nil)
})
pub fn with_snapshot(
db: Db(k, v),
callback callback: fn(Snapshot(k, v)) -> a,
) -> a
Run a callback with a point-in-time snapshot. The snapshot sees the state of the database at the moment it was acquired — subsequent writes are invisible to it.
Non-escaping: The Snapshot handle is only valid inside the callback.
Do not store it in a variable, send it to another process, or return it —
using a Snapshot after the callback returns will panic or produce
undefined behavior because the underlying file handle is closed on exit.
Panics if the snapshot file handle cannot be opened.
let result = trove.with_snapshot(db, fn(snap) {
trove.snapshot_get(snapshot: snap, key: "my_key")
})
// result: Result(String, Nil)