RocksDB Statistics Guide

This guide explains how to collect and use RocksDB statistics in Erlang RocksDB. Statistics provide insights into database performance, I/O operations, cache efficiency, and more.

Getting Started

Creating a Statistics Object

First, create a statistics object and attach it to your database:

%% Create a statistics collector
{ok, Stats} = rocksdb:new_statistics(),

%% Open database with statistics enabled
{ok, Db} = rocksdb:open("my_db", [
    {create_if_missing, true},
    {statistics, Stats}
]).

Reading Statistics

Use rocksdb:statistics_ticker/2 to read counter values:

{ok, KeysWritten} = rocksdb:statistics_ticker(Stats, number_keys_written),
io:format("Keys written: ~p~n", [KeysWritten]).

Use rocksdb:statistics_histogram/2 to read histogram data:

{ok, WriteHist} = rocksdb:statistics_histogram(Stats, db_write),
io:format("Write latency - median: ~.2f us, p99: ~.2f us~n",
          [maps:get(median, WriteHist), maps:get(percentile99, WriteHist)]).

Histogram Data Format

Histogram results are returned as a map with the following keys:

Setting Statistics Level

Control the level of detail collected:

%% Set to collect all statistics
ok = rocksdb:set_stats_level(Stats, stats_all),

%% Or disable expensive timing stats
ok = rocksdb:set_stats_level(Stats, stats_except_timers).

Available levels:

• stats_disable_all - Disable all statistics
• stats_except_tickers - Collect histograms only
• stats_except_histogram_or_timers - Collect tickers only (no histograms or timing)
• stats_except_timers - Collect everything except timing measurements
• stats_except_detailed_timers - Collect everything except detailed timing
• stats_except_time_for_mutex - Collect everything except mutex timing
• stats_all - Collect all statistics (default)

Cleanup

Release the statistics object when done:

ok = rocksdb:close(Db),
ok = rocksdb:release_statistics(Stats).

Complete Ticker Reference

Tickers are simple counters that track cumulative values.

Database Operation Tickers

Iterator Tickers

Block Cache Tickers

Example - Calculate cache hit ratio:

{ok, Hits} = rocksdb:statistics_ticker(Stats, block_cache_hit),
{ok, Misses} = rocksdb:statistics_ticker(Stats, block_cache_miss),
HitRatio = case Hits + Misses of
    0 -> 0.0;
    Total -> Hits / Total * 100
end,
io:format("Block cache hit ratio: ~.2f%~n", [HitRatio]).

Memtable Tickers

Write Path Tickers

Compaction Tickers

BlobDB Tickers

Transaction Tickers

Complete Histogram Reference

Histograms track the distribution of values over time.

Database Operation Histograms

Example - Monitor read latency:

{ok, GetHist} = rocksdb:statistics_histogram(Stats, db_get),
io:format("Get latency:~n"),
io:format("  Median: ~.2f us~n", [maps:get(median, GetHist)]),
io:format("  P95:    ~.2f us~n", [maps:get(percentile95, GetHist)]),
io:format("  P99:    ~.2f us~n", [maps:get(percentile99, GetHist)]),
io:format("  Max:    ~.2f us~n", [maps:get(max, GetHist)]).

Compaction and Flush Histograms

I/O Histograms

BlobDB Histograms

Transaction Histograms

Example: Comprehensive Monitoring

Here's a complete example that monitors key database metrics:

-module(db_monitor).
-export([report/1]).

report(Stats) ->
    %% Operation counts
    {ok, KeysWritten} = rocksdb:statistics_ticker(Stats, number_keys_written),
    {ok, KeysRead} = rocksdb:statistics_ticker(Stats, number_keys_read),

    %% Cache efficiency
    {ok, CacheHits} = rocksdb:statistics_ticker(Stats, block_cache_hit),
    {ok, CacheMisses} = rocksdb:statistics_ticker(Stats, block_cache_miss),
    CacheHitRatio = safe_ratio(CacheHits, CacheHits + CacheMisses),

    %% Memtable efficiency
    {ok, MemHits} = rocksdb:statistics_ticker(Stats, memtable_hit),
    {ok, MemMisses} = rocksdb:statistics_ticker(Stats, memtable_miss),
    MemHitRatio = safe_ratio(MemHits, MemHits + MemMisses),

    %% Write stalls
    {ok, StallMicros} = rocksdb:statistics_ticker(Stats, stall_micros),

    %% Latency histograms
    {ok, GetHist} = rocksdb:statistics_histogram(Stats, db_get),
    {ok, WriteHist} = rocksdb:statistics_histogram(Stats, db_write),

    io:format("=== RocksDB Statistics ===~n"),
    io:format("Keys written: ~p, Keys read: ~p~n", [KeysWritten, KeysRead]),
    io:format("Block cache hit ratio: ~.2f%~n", [CacheHitRatio * 100]),
    io:format("Memtable hit ratio: ~.2f%~n", [MemHitRatio * 100]),
    io:format("Total stall time: ~.2f ms~n", [StallMicros / 1000]),
    io:format("Get latency p99: ~.2f us~n", [maps:get(percentile99, GetHist)]),
    io:format("Write latency p99: ~.2f us~n", [maps:get(percentile99, WriteHist)]),
    ok.

safe_ratio(_, 0) -> 0.0;
safe_ratio(Num, Denom) -> Num / Denom.

Statistics with Column Families

When using column families, create one statistics object and share it:

{ok, Stats} = rocksdb:new_statistics(),
{ok, Db, [DefaultCf, DataCf, IndexCf]} = rocksdb:open_with_cf(
    "my_db",
    [{create_if_missing, true}, {statistics, Stats}],
    [{"default", []}, {"data", []}, {"index", []}]
),

%% Statistics are aggregated across all column families
{ok, TotalKeysWritten} = rocksdb:statistics_ticker(Stats, number_keys_written).

Statistics with Transactions

Statistics also work with pessimistic transaction databases:

{ok, Stats} = rocksdb:new_statistics(),
{ok, Db, _} = rocksdb:open_pessimistic_transaction_db(
    "txn_db",
    [{create_if_missing, true}, {statistics, Stats}],
    [{"default", []}]
),

%% Perform transactions...
{ok, Txn} = rocksdb:pessimistic_transaction(Db, []),
ok = rocksdb:pessimistic_transaction_put(Txn, <<"key">>, <<"value">>),
ok = rocksdb:pessimistic_transaction_commit(Txn),
ok = rocksdb:release_pessimistic_transaction(Txn),

%% Check transaction-specific stats
{ok, OpsPerTxn} = rocksdb:statistics_histogram(Stats, num_op_per_transaction),
io:format("Avg ops per transaction: ~.2f~n", [maps:get(average, OpsPerTxn)]).

Performance Considerations

1. Statistics overhead: Collecting statistics has a small performance cost. Use stats_except_timers or stats_except_detailed_timers in production if timing precision isn't critical.

2. Polling frequency: Statistics are cumulative. Poll them periodically and compute deltas for rate metrics.

3. Memory: The statistics object uses minimal memory but should be released when no longer needed.

4. Thread safety: Statistics are thread-safe and can be read from any Erlang process.