shelf

Persistent ETS tables backed by DETS — fast in-memory access with automatic disk persistence for the BEAM.

shelf is not yet 1.0. This means:

• the API is unstable
• features and APIs may be removed in minor releases
• quality should not be considered production-ready

We welcome usage and feedback in the meantime! We will do our best to minimize breaking changes regardless.

Shelf combines ETS (fast, in-memory) with DETS (persistent, on-disk) to give you microsecond reads with durable storage. It implements the classic Erlang persistence pattern, wrapped in a type-safe Gleam API.

If you only need ETS or DETS individually, check out these excellent standalone wrappers:

• bravo — Type-safe ETS wrapper for Gleam
• slate — Type-safe DETS wrapper for Gleam

Shelf coordinates both together, using Erlang’s native ets:to_dets/2 and ets:from_dets/2 for efficient bulk transfers between the two.

Quick Start

gleam add shelf

import shelf
import shelf/set

pub fn main() {
  // Open a persistent set — loads existing data from disk
  let assert Ok(table) = set.open(name: "users", path: "data/users.dets")

  // Fast writes (to ETS)
  let assert Ok(Nil) = set.insert(table, "alice", 42)
  let assert Ok(Nil) = set.insert(table, "bob", 99)

  // Fast reads (from ETS)
  let assert Ok(42) = set.lookup(table, "alice")

  // Persist to disk when ready
  let assert Ok(Nil) = set.save(table)

  // Close auto-saves
  let assert Ok(Nil) = set.close(table)
}

On next startup, set.open automatically loads the saved data back into ETS.

How It Works

┌─────────────────────────────────────┐
│           Your Application          │
├─────────────────────────────────────┤
│         shelf (this library)        │
├──────────────────┬──────────────────┤
│    ETS (memory)  │   DETS (disk)    │
│  • μs reads      │  • persistence   │
│  • μs writes     │  • survives      │
│  • in-process    │    restarts      │
└──────────────────┴──────────────────┘

Reads always go to ETS — consistent microsecond latency regardless of table size.

Writes go to ETS immediately. When they hit DETS depends on the write mode:

Write Modes

WriteBack (default)

Writes go to ETS only. You control when to persist:

let assert Ok(table) = set.open(name: "sessions", path: "data/sessions.dets")

// These are ETS-only (fast)
set.insert(table, "user:123", session)
set.insert(table, "user:456", session)

// Persist when ready (e.g., on a timer, after N writes)
set.save(table)

// Undo unsaved changes
set.reload(table)

WriteThrough

Every write persists immediately:

let config =
  shelf.config(name: "accounts", path: "data/accounts.dets")
  |> shelf.write_mode(shelf.WriteThrough)

let assert Ok(table) = set.open_config(config)

// This writes to both ETS and DETS
set.insert(table, "acct:789", account)

Table Types

Set — unique keys

import shelf/set

let assert Ok(t) = set.open(name: "cache", path: "cache.dets")
set.insert(t, "key", "value")       // overwrites if exists
set.insert_new(t, "key", "value2")  // fails if exists
set.lookup(t, "key")                // Ok("value")

Bag — multiple distinct values per key

import shelf/bag

let assert Ok(t) = bag.open(name: "tags", path: "tags.dets")
bag.insert(t, "color", "red")
bag.insert(t, "color", "blue")
bag.insert(t, "color", "red")    // ignored (duplicate)
bag.lookup(t, "color")           // Ok(["red", "blue"])

Duplicate Bag — duplicates allowed

import shelf/duplicate_bag

let assert Ok(t) = duplicate_bag.open(name: "events", path: "events.dets")
duplicate_bag.insert(t, "click", "btn")
duplicate_bag.insert(t, "click", "btn")  // kept!
duplicate_bag.lookup(t, "click")         // Ok(["btn", "btn"])

Safe Resource Management

Use with_table to ensure tables are always closed:

use table <- set.with_table("cache", "data/cache.dets")
set.insert(table, "key", "value")
// table is auto-closed when the callback returns

Persistence Operations

Atomic Counters

let assert Ok(t) = set.open(name: "stats", path: "stats.dets")
set.insert(t, "page_views", 0)
set.update_counter(t, "page_views", 1)   // Ok(1)
set.update_counter(t, "page_views", 10)  // Ok(11)

Limitations

• DETS file size: 2 GB maximum per table
• No ordered set: DETS doesn’t support ordered_set
• Erlang only: Requires the BEAM runtime (no JavaScript target)
• Single node: DETS is local to one node (use Mnesia for distribution)
• Table names: Must be unique across all ETS tables in the VM

See Also

• bravo — Use ETS directly when you don’t need disk persistence
• slate — Use DETS directly when you don’t need in-memory speed
• Erlang ETS docs — Underlying ETS documentation
• Erlang DETS docs — Underlying DETS documentation

Target

This package only supports the Erlang target.

Development

gleam test    # Run the test suite
gleam build   # Build the package
gleam format  # Format source code

Further documentation can be found at https://hexdocs.pm/shelf.