Veidrodelis (veidrodelis v0.1.6)

Copy Markdown View Source

This is the main module for managing in-process replicas of Redis/Valkey data.

This module provides a simple interface for starting/stopping/inspecting replica instances, as well as commands for accessing the replicated data.

Veidrodelis instance is a single GenServer process that connects to Redis/Valkey, fetches the data via replication protocol and builds an in-memory projection of the data.

The process is registered under global id which is used for reading the replicated data without calls to the replicating GenServer.

Sample Usage

  # Start a Veidrodelis instance
  {:ok, pid} = Veidrodelis.start_link(
    id: :my_instance,
    host: "localhost",
    port: 6379
  )

  # Query data using accessor functions
  {:ok, value} = Veidrodelis.get(:my_instance, 0, "mykey")
  {:ok, len} = Veidrodelis.llen(:my_instance, 0, "mylist")

  # Read ransacitons
  {:ok, [{:ok, debit}, {:ok, credit}]} = Veidrodelis.read_tx(:my_instance, 0, [
    {:get, "account:123:debit"},
    {:get, "account:123:credit"}
  ])

  # Read transactions via Lua script
  {:ok, [debit, credit]} = Veidrodelis.read_tx(
    :my_instance,
    0,
    "return {ts.get('account:123:debit'), ts.get('account:123:credit')}"
  )

  # Stop the instance
  :ok = Veidrodelis.stop(pid)

Summary

Functions

Gets the value of a string key.

Gets the host and port of the currently connected Redis server.

Gets the current replication state of a Veidrodelis instance.

Returns true if field exists in the hash stored at key.

Returns up to count lexicographically first field/value pairs in the hash stored at key.

Returns the value associated with field in the hash stored at key.

Returns all fields and values of the hash stored at key.

Returns all field names in the hash stored at key.

Returns up to count lexicographically last field/value pairs in reverse order.

Returns the number of fields in the hash stored at key.

Returns the values associated with the specified fields in the hash stored at key.

Returns up to count field/value pairs after field in the hash stored at key.

Returns up to count field/value pairs before field in the hash stored at key.

Returns up to count random fields (and optional values) from the hash stored at key.

Returns the length of the value stored at field (string length).

Returns all values in the hash stored at key.

Returns the length of the list stored at key.

Returns the specified elements of the list stored at key.

Compiles a Lua script to bytecode for faster execution.

Executes multiple read-only commands or a Lua script atomically.

Returns the cardinality (number of elements) of the set stored at key.

Returns the difference of the provided sets (first key minus remaining keys).

Gets up to count members from the beginning of a set.

Returns the intersection of the provided sets.

Returns the cardinality of the intersection of the provided sets.

Returns whether member is a member of the set stored at key.

Gets up to count members from the end of a set in reverse order.

Returns all members of the set stored at key.

Returns membership status for each member in the provided list.

Gets up to count members after the given member in a set.

Gets up to count members before the given member in a set.

Returns up to count random members from the set stored at key.

Starts a Veidrodelis instance that connects to Redis and processes replication stream.

Stops a Veidrodelis instance.

Returns the union of the provided sets.

Unsubscribes from updates for a specific key.

Subscribes to updates for a specific key in a database.

Returns the cardinality (number of elements) of the sorted set stored at key.

Returns up to count member/score pairs from the start of the sorted set.

Returns up to count member/score pairs from the end of the sorted set in reverse order.

Returns up to count member/score pairs after member in the sorted set.

Returns up to count member/score pairs before member in the sorted set.

Returns the specified range of elements in the sorted set stored at key.

Returns all members in the sorted set stored at key with scores between min and max (inclusive).

Returns the rank (0-based index) where the member would be in the sorted set, ordered from lowest to highest score. Returns nil if the member or key doesn't exist.

Returns the rank (0-based index) where the member would be in the sorted set, ordered from highest to lowest score. Returns nil if the member or key doesn't exist.

Returns the score of member in the sorted set stored at key.

Types

command()

@type command() :: tuple()

db()

@type db() :: non_neg_integer()

hash_key()

@type hash_key() :: binary()

instance_id()

@type instance_id() :: term()

key()

@type key() :: binary()

lua_compiled_script()

@type lua_compiled_script() :: binary()

lua_script()

@type lua_script() :: binary()

score()

@type score() :: float()

value()

@type value() :: binary()

Functions

get(id, db, key)

@spec get(instance_id(), db(), key()) :: {:ok, binary() | nil} | {:error, term()}

Gets the value of a string key.

get_connected_to(pid)

@spec get_connected_to(pid() | instance_id()) ::
  {:ok, {String.t(), non_neg_integer()}} | {:error, :not_connected}

Gets the host and port of the currently connected Redis server.

Returns the actual host and port that the Veidrodelis instance is connected to. For sentinel-based connections, this returns the discovered server address. For direct connections, this returns the configured host and port.

Parameters

  • pid_or_id - Either the Replica GenServer PID or the Veidrodelis instance ID

Returns

  • {:ok, {host, port}} - The connected host (string) and port (integer)
  • {:error, :not_connected} - Replica is not currently connected

Examples

# Using instance ID
{:ok, {host, port}} = Veidrodelis.get_connected_to(:my_instance)

# Using PID directly
{:ok, pid} = Veidrodelis.start_link(id: :test, host: "localhost", port: 6379)
{:ok, {host, port}} = Veidrodelis.get_connected_to(pid)

get_replication_state(pid)

@spec get_replication_state(pid() | instance_id()) ::
  Vdr.RedisStream.Replica.replica_state()

Gets the current replication state of a Veidrodelis instance.

hexists(id, db, key, field)

@spec hexists(instance_id(), db(), key(), hash_key()) ::
  {:ok, boolean()} | {:error, term()}

Returns true if field exists in the hash stored at key.

hfirst(id, db, key, count)

@spec hfirst(instance_id(), db(), key(), non_neg_integer()) ::
  {:ok, [{hash_key(), binary()}]} | {:error, term()}

Returns up to count lexicographically first field/value pairs in the hash stored at key.

hget(id, db, key, field)

@spec hget(instance_id(), db(), key(), hash_key()) ::
  {:ok, binary() | nil} | {:error, term()}

Returns the value associated with field in the hash stored at key.

hgetall(id, db, key)

@spec hgetall(instance_id(), db(), key()) ::
  {:ok, [{hash_key(), binary()}]} | {:error, term()}

Returns all fields and values of the hash stored at key.

hkeys(id, db, key)

@spec hkeys(instance_id(), db(), key()) :: {:ok, [hash_key()]} | {:error, term()}

Returns all field names in the hash stored at key.

hlast(id, db, key, count)

@spec hlast(instance_id(), db(), key(), non_neg_integer()) ::
  {:ok, [{hash_key(), binary()}]} | {:error, term()}

Returns up to count lexicographically last field/value pairs in reverse order.

hlen(id, db, key)

@spec hlen(instance_id(), db(), key()) :: {:ok, non_neg_integer()} | {:error, term()}

Returns the number of fields in the hash stored at key.

hmget(id, db, key, fields)

@spec hmget(instance_id(), db(), key(), [hash_key()]) ::
  {:ok, [binary() | nil]} | {:error, term()}

Returns the values associated with the specified fields in the hash stored at key.

hnext(id, db, key, field, count)

@spec hnext(instance_id(), db(), key(), binary(), non_neg_integer()) ::
  {:ok, [{hash_key(), binary()}]} | {:error, term()}

Returns up to count field/value pairs after field in the hash stored at key.

hprev(id, db, key, field, count)

@spec hprev(instance_id(), db(), key(), binary(), non_neg_integer()) ::
  {:ok, [{hash_key(), binary()}]} | {:error, term()}

Returns up to count field/value pairs before field in the hash stored at key.

hrandfield(id, db, key, count, with_values \\ true)

@spec hrandfield(instance_id(), db(), key(), integer(), boolean()) ::
  {:ok, [{hash_key(), binary()} | binary()]} | {:error, term()}

Returns up to count random fields (and optional values) from the hash stored at key.

hstrlen(id, db, key, field)

@spec hstrlen(instance_id(), db(), key(), hash_key()) ::
  {:ok, non_neg_integer()} | {:error, term()}

Returns the length of the value stored at field (string length).

hvals(id, db, key)

@spec hvals(instance_id(), db(), key()) :: {:ok, [binary()]} | {:error, term()}

Returns all values in the hash stored at key.

llen(id, db, key)

@spec llen(instance_id(), db(), key()) :: {:ok, non_neg_integer()} | {:error, term()}

Returns the length of the list stored at key.

lrange(id, db, key, start_idx, stop_idx)

@spec lrange(instance_id(), db(), key(), integer(), integer()) ::
  {:ok, [binary()]} | {:error, term()}

Returns the specified elements of the list stored at key.

lua_load(id, script)

@spec lua_load(instance_id(), lua_script()) ::
  {:ok, lua_compiled_script()} | {:error, term()}

Compiles a Lua script to bytecode for faster execution.

The compiled bytecode can be passed to read_tx/3 instead of a script string. This is useful when the same script will be executed multiple times.

Examples

{:ok, pid} = Veidrodelis.start_link(id: :my_instance)
script = "return ts.get('key1')"
{:ok, bytecode} = Veidrodelis.lua_load(:my_instance, script)
{:ok, result} = Veidrodelis.read_tx(:my_instance, 0, bytecode)

read_tx(id, db, commands)

@spec read_tx(instance_id(), db(), [command()] | lua_script() | lua_compiled_script()) ::
  {:ok, term() | [ok: term(), error: term()]} | {:error, term()}

Executes multiple read-only commands or a Lua script atomically.

With a list of commands

Executes multiple read commands atomically. Commands are executed in order and their results are returned as a list of tuples.

Supported Commands

  • {:get, key} - Get string value
  • {:hget, key, field} - Get hash field value
  • {:hmget, key, fields} - Get multiple hash field values
  • {:hgetall, key} - Get all hash fields and values
  • {:hkeys, key} - Get hash field names
  • {:hvals, key} - Get hash values
  • {:hlen, key} - Get hash length
  • {:hexists, key, field} - Check hash field existence
  • {:hstrlen, key, field} - Get hash field length
  • {:hrandfield, key, count, with_values} - Random hash field(s)
  • {:hfirst, key, count} - Get up to count field/value pairs from the beginning
  • {:hlast, key, count} - Get up to count field/value pairs from the end
  • {:hnext, key, field, count} - Get up to count field/value pairs after field
  • {:hprev, key, field, count} - Get up to count field/value pairs before field
  • {:llen, key} - Get list length
  • {:lrange, key, start, stop} - Get list range
  • {:smembers, key} - Get set members
  • {:sismember, key, member} - Check set membership
  • {:scard, key} - Get set cardinality
  • {:smismember, key, members} - Check membership of multiple members
  • {:srandmember, key, count} - Get random members
  • {:sunion, keys} - Union of sets
  • {:sinter, keys} - Intersection of sets
  • {:sdiff, keys} - Difference of sets
  • {:sintercard, keys} - Cardinality of intersection
  • {:sfirst, key, count} - Get up to count members from the beginning
  • {:slast, key, count} - Get up to count members from the end
  • {:snext, key, member, count} - Get up to count members immediately after member
  • {:sprev, key, member, count} - Get up to count members immediately before member
  • {:zscore, key, member} - Get sorted set member score
  • {:zcard, key} - Get sorted set cardinality
  • {:zrange, key, start, stop, with_scores} - Get sorted set range
  • {:zrangebyscore, key, min, max, with_scores} - Get sorted set range by score
  • {:zrank, key, member} - Get sorted set member rank
  • {:zrevrank, key, member} - Get sorted set member reverse rank
  • {:zcount, key, min, max} - Count sorted set members in score range
  • {:zfirst, key, count} - Get up to count member/score pairs from the beginning
  • {:zlast, key, count} - Get up to count member/score pairs from the end
  • {:znext, key, member, count} - Get up to count member/score pairs after member
  • {:zprev, key, member, count} - Get up to count member/score pairs before member

Example

  {:ok, [value1, value2]} = Veidrodelis.read_tx(:my_instance, 0, [
    {:get, "key1"},
    {:hget, "hash1", "field1"}
  ])

With a Lua script

Executes a Lua script atomically under the storage mutex. The script has access to:

  • ts.get(key) - Get a string value
  • ts.hget(key, field) - Get a hash field value
  • ts.llen(key) - Get list length
  • ts.lrange(key, start, stop) - Get list range
  • ts.smembers(key) - Get set members
  • ts.sismember(key, member) - Check set membership
  • ts.scard(key) - Get set cardinality
  • ts.smismember(key, members) - Check multiple members
  • ts.srandmember(key, count) - Get random members
  • ts.sunion(keys) - Union of sets
  • ts.sinter(keys) - Intersection of sets
  • ts.sdiff(keys) - Difference of sets
  • ts.sintercard(keys) - Intersection cardinality
  • ts.sfirst(key, count) - Get first set members
  • ts.slast(key, count) - Get last set members
  • ts.snext(key, member, count) - Get next set members
  • ts.sprev(key, member, count) - Get previous set members
  • ts.hgetall(key) - Get all hash fields and values
  • ts.hmget(key, fields) - Get multiple hash field values
  • ts.hkeys(key) - Get hash field names
  • ts.hvals(key) - Get hash values
  • ts.hlen(key) - Get hash length
  • ts.hexists(key, field) - Check if hash field exists
  • ts.hstrlen(key, field) - Get hash field length
  • ts.hrandfield(key, count, with_values) - Get random hash fields
  • ts.hfirst(key, count) - Get first hash field/value pairs
  • ts.hlast(key, count) - Get last hash field/value pairs
  • ts.hnext(key, field, count) - Get next hash fields after given field
  • ts.hprev(key, field, count) - Get previous hash fields before given field
  • ts.zscore(key, member) - Get sorted set member score
  • ts.zcard(key) - Get sorted set cardinality
  • ts.zrange(key, start, stop) - Get sorted set range
  • ts.zrangebyscore(key, min, max) - Get sorted set range by score
  • ts.zrank(key, member) - Get sorted set member rank
  • ts.zrevrank(key, member) - Get sorted set member reverse rank
  • ts.zcount(key, min, max) - Count sorted set members in score range
  • ts.zfirst(key, count) - Get first sorted set members
  • ts.zlast(key, count) - Get last sorted set members
  • ts.znext(key, member, count) - Get next sorted set members
  • ts.zprev(key, member, count) - Get previous sorted set members

Example

{:ok, pid} = Veidrodelis.start_link(id: :my_instance)
script = "return ts.get('key1')"
{:ok, result} = Veidrodelis.read_tx(:my_instance, 0, script)

Returns

  • {:ok, results} - For commands: list of results. For scripts: the script's return value
  • {:error, reason} - Execution error

scard(id, db, key)

@spec scard(instance_id(), db(), key()) :: {:ok, non_neg_integer()} | {:error, term()}

Returns the cardinality (number of elements) of the set stored at key.

sdiff(id, db, keys)

@spec sdiff(instance_id(), db(), [key()]) :: {:ok, [binary()]} | {:error, term()}

Returns the difference of the provided sets (first key minus remaining keys).

sfirst(id, db, key, count)

@spec sfirst(instance_id(), db(), key(), non_neg_integer()) ::
  {:ok, [binary()]} | {:error, term()}

Gets up to count members from the beginning of a set.

sinter(id, db, keys)

@spec sinter(instance_id(), db(), [key()]) :: {:ok, [binary()]} | {:error, term()}

Returns the intersection of the provided sets.

sintercard(id, db, keys)

@spec sintercard(instance_id(), db(), [key()]) ::
  {:ok, non_neg_integer()} | {:error, term()}

Returns the cardinality of the intersection of the provided sets.

sismember(id, db, key, member)

@spec sismember(instance_id(), db(), key(), binary()) ::
  {:ok, boolean()} | {:error, term()}

Returns whether member is a member of the set stored at key.

slast(id, db, key, count)

@spec slast(instance_id(), db(), key(), non_neg_integer()) ::
  {:ok, [binary()]} | {:error, term()}

Gets up to count members from the end of a set in reverse order.

smembers(id, db, key)

@spec smembers(instance_id(), db(), key()) :: {:ok, [binary()]} | {:error, term()}

Returns all members of the set stored at key.

smismember(id, db, key, members)

@spec smismember(instance_id(), db(), key(), [binary()]) ::
  {:ok, [boolean()]} | {:error, term()}

Returns membership status for each member in the provided list.

snext(id, db, key, member, count)

@spec snext(instance_id(), db(), key(), binary(), non_neg_integer()) ::
  {:ok, [binary()]} | {:error, term()}

Gets up to count members after the given member in a set.

sprev(id, db, key, member, count)

@spec sprev(instance_id(), db(), key(), binary(), non_neg_integer()) ::
  {:ok, [binary()]} | {:error, term()}

Gets up to count members before the given member in a set.

srandmember(id, db, key, count)

@spec srandmember(instance_id(), db(), key(), integer()) ::
  {:ok, [binary()]} | {:error, term()}

Returns up to count random members from the set stored at key.

start_link(opts)

@spec start_link(keyword()) :: {:ok, pid()} | {:error, term()}

Starts a Veidrodelis instance that connects to Redis and processes replication stream.

Options

  • :id - Veidrodelis instance ID, required
  • :host - Redis host (default: "localhost"). Cannot be used with :sentinel.
  • :port - Redis port (default: 6379). Cannot be used with :sentinel.
  • :sentinel - Sentinel configuration (keyword list). Cannot be used with :host/:port.
    • :sentinels - List of sentinel nodes (required), each with :host and :port
    • :group - Name of the primary group in sentinel (required)
    • :role - Server role to discover: :primary or :replica (default: :primary)
    • :connect_opts - Redix connection options for sentinel connections (optional)
    • :replica_connect_opts - Redix connection options for discovered Redis server (optional)
  • :username - Redis username for ACL authentication (default: nil). Only for direct connections.
  • :password - Redis password (default: nil). Only for direct connections.
  • :ssl - Use SSL/TLS (default: false). Only for direct connections.
  • :ssl_opts - SSL options (default: []). Only for direct connections.
  • :reconnect - Enable automatic reconnection (default: true)
  • :reconnect_delay_ms - Initial delay before reconnection in ms (default: 1000)
  • :max_reconnect_delay_ms - Maximum delay between reconnection attempts in ms (default: 30000)

For full documentation of sentinel options and advanced features, see Vdr.RedisStream.Replica.

Returns

  • {:ok, pid} - Successfully started (returns Replica GenServer PID)
  • {:error, reason} - Failed to start

Examples

Direct Connection

opts = [
  id: :my_instance,
  host: "localhost",
  port: 6379
]
{:ok, pid} = Veidrodelis.start_link(opts)

Sentinel Connection

opts = [
  id: :my_instance,
  sentinel: [
    sentinels: [
      [host: "sentinel1", port: 26379],
      [host: "sentinel2", port: 26379]
    ],
    group: "myprimary",
    role: :primary,
    # Optional: Connection options for sentinel
    connect_opts: [timeout: 1000],
    # Optional: Connection options for Redis server
    replica_connect_opts: [password: "redis_password", ssl: true]
  ]
]
{:ok, pid} = Veidrodelis.start_link(opts)

stop(pid)

@spec stop(pid()) :: :ok

Stops a Veidrodelis instance.

sunion(id, db, keys)

@spec sunion(instance_id(), db(), [key()]) :: {:ok, [binary()]} | {:error, term()}

Returns the union of the provided sets.

unwatch(id, db, key, timeout \\ 5000)

Unsubscribes from updates for a specific key.

Parameters

  • id - Veidrodelis instance ID
  • db - Database number
  • key - The key to unwatch (binary)
  • timeout - The timeout for the call (default: 5000ms)

Returns

  • :ok - Successfully unsubscribed
  • {:error, :not_found} - Instance or watch not found

Example

# Unsubscribe from key updates
:ok = Veidrodelis.unwatch(:my_instance, 0, "user:123")

watch(id, db, key, ref, timeout \\ 5000)

Subscribes to updates for a specific key in a database.

When the key is modified, the calling process will receive a message: {ref, %Vdr.WatchEvent.Update{command: cmd, db: db}}

When the replica transitions to streaming mode after an RDB transfer, the calling process will receive: {ref, %Vdr.WatchEvent.Init{}}

Parameters

  • id - Veidrodelis instance ID
  • db - Database number
  • key - The key to watch (binary)
  • ref - Reference value to identify this watch in notifications
  • timeout - The timeout for the call (default: 5000ms)

Returns

  • :ok - Successfully subscribed
  • {:error, :not_found} - Instance not found
  • {:error, :already_registered} - This process is already watching this key

Example

# Subscribe to key updates
:ok = Veidrodelis.watch(:my_instance, 0, "user:123", :my_watch_ref)

# Receive notifications
receive do
  {:my_watch_ref, %Vdr.WatchEvent.Update{command: cmd, db: db}} ->
    IO.inspect({:update, cmd, db})

  {:my_watch_ref, %Vdr.WatchEvent.Init{}} ->
    IO.puts("Streaming mode started")
end

zcard(id, db, key)

@spec zcard(instance_id(), db(), key()) :: {:ok, non_neg_integer()} | {:error, term()}

Returns the cardinality (number of elements) of the sorted set stored at key.

zfirst(id, db, key, count)

@spec zfirst(instance_id(), db(), key(), non_neg_integer()) ::
  {:ok, [{score(), binary()}]} | {:error, term()}

Returns up to count member/score pairs from the start of the sorted set.

zlast(id, db, key, count)

@spec zlast(instance_id(), db(), key(), non_neg_integer()) ::
  {:ok, [{score(), binary()}]} | {:error, term()}

Returns up to count member/score pairs from the end of the sorted set in reverse order.

znext(id, db, key, member, count)

@spec znext(instance_id(), db(), key(), binary(), non_neg_integer()) ::
  {:ok, [{score(), binary()}]} | {:error, term()}

Returns up to count member/score pairs after member in the sorted set.

zprev(id, db, key, member, count)

@spec zprev(instance_id(), db(), key(), binary(), non_neg_integer()) ::
  {:ok, [{score(), binary()}]} | {:error, term()}

Returns up to count member/score pairs before member in the sorted set.

zrange(id, db, key, start_idx, stop_idx, with_scores \\ true)

@spec zrange(instance_id(), db(), key(), integer(), integer(), boolean()) ::
  {:ok, [{binary(), score()}] | [binary()]} | {:error, term()}

Returns the specified range of elements in the sorted set stored at key.

If with_scores is true, returns a list of {member, score} tuples. If with_scores is false, returns a list of members only.

zrangebyscore(id, db, key, min, max, with_scores \\ true)

@spec zrangebyscore(instance_id(), db(), key(), score(), score(), boolean()) ::
  {:ok, [{binary(), score()}] | [binary()]} | {:error, term()}

Returns all members in the sorted set stored at key with scores between min and max (inclusive).

If with_scores is true, returns a list of {member, score} tuples. If with_scores is false, returns a list of members only.

Examples

{:ok, pid} = Veidrodelis.start_link(id: :my_instance)
{:ok, members} = Veidrodelis.zrangebyscore(:my_instance, 0, "myzset", 1.0, 3.0, true)
# Returns: [{"one", 1.0}, {"two", 2.0}, {"three", 3.0}]

{:ok, members} = Veidrodelis.zrangebyscore(:my_instance, 0, "myzset", 1.0, 3.0, false)
# Returns: ["one", "two", "three"]

zrank(id, db, key, member)

@spec zrank(instance_id(), db(), key(), binary()) ::
  {:ok, non_neg_integer() | nil} | {:error, term()}

Returns the rank (0-based index) where the member would be in the sorted set, ordered from lowest to highest score. Returns nil if the member or key doesn't exist.

zrevrank(id, db, key, member)

@spec zrevrank(instance_id(), db(), key(), binary()) ::
  {:ok, non_neg_integer() | nil} | {:error, term()}

Returns the rank (0-based index) where the member would be in the sorted set, ordered from highest to lowest score. Returns nil if the member or key doesn't exist.

zscore(id, db, key, member)

@spec zscore(instance_id(), db(), key(), any()) ::
  {:ok, score() | nil} | {:error, term()}

Returns the score of member in the sorted set stored at key.