Nebulex.Adapters.Redis (Nebulex.Adapters.Redis v3.0.0-rc.2)

Nebulex adapter for Redis. This adapter is implemented using Redix (a Redis driver for Elixir).

The adapter provides three setup alternatives:

Standalone - The adapter establishes a pool of connections with a single Redis node. The :standalone is the default mode.
Redis Cluster - Redis Cluster is a built-in feature in Redis since version 3, and it may be the most convenient and recommendable way to set up Redis in a cluster and have a distributed cache storage out-of-box. This adapter provides the :redis_cluster mode to set up Redis Cluster from the client-side automatically and be able to use it transparently.
Built-in client-side cluster - The :client_side_cluster mode provides a simple client-side cluster implementation based on sharding distribution model.

Standalone

A cache that uses Redis is defined as follows:

defmodule MyApp.RedisCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: Nebulex.Adapters.Redis
end

The configuration for the cache must be in your application environment, usually defined in your config/config.exs:

config :my_app, MyApp.RedisCache,
  conn_opts: [
    host: "127.0.0.1",
    port: 6379
  ]

Redis Cluster

A cache that uses Redis Cluster can be defined as follows:

defmodule MyApp.RedisClusterCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: Nebulex.Adapters.Redis
end

As you may notice, nothing has changed, it is defined the same as the standalone mode. The change is in the configuration:

config :my_app, MyApp.RedisClusterCache,
  mode: :redis_cluster,
  redis_cluster: [
    # Configuration endpoints
    # The client connects to these endpoints to fetch cluster topology
    # using "CLUSTER SHARDS" (Redis >= 7) or "CLUSTER SLOTS" (Redis < 7).
    # Multiple endpoints can be provided for redundancy. The adapter will
    # try each endpoint until it successfully retrieves the topology.
    configuration_endpoints: [
      endpoint1_conn_opts: [
        host: "127.0.0.1",
        port: 6379,
        # Add the password if 'requirepass' is enabled
        password: "password"
      ],
      endpoint2_conn_opts: [
        host: "127.0.0.1",
        port: 6380,
        password: "password"
      ]
    ],

    # Optional: Override master host addresses returned by the cluster.
    # Useful when running Redis in Docker or behind NAT where the
    # advertised addresses differ from the actual connection addresses.
    # Defaults to `false`.
    override_master_host: false
  ]

Client-side Cluster

Same as the previous modes, a cache is defined as:

defmodule MyApp.ClusteredCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: Nebulex.Adapters.Redis
end

The configuration:

config :my_app, MyApp.ClusteredCache,
  mode: :client_side_cluster,
  client_side_cluster: [
    nodes: [
      node1: [
        pool_size: 10,
        conn_opts: [
          host: "127.0.0.1",
          port: 9001
        ]
      ],
      node2: [
        pool_size: 4,
        conn_opts: [
          url: "redis://127.0.0.1:9002"
        ]
      ],
      node3: [
        conn_opts: [
          host: "127.0.0.1",
          port: 9003
        ]
      ],
      ...
    ]
  ]

Redis Proxy Alternative

Consider using a proxy instead, since it may provide more and better features. See the "Redis Proxy" section below for more information.

Redis Proxy

Another option for "Redis Cluster" or the built-in "Client-side cluster" is using a proxy such as Envoy proxy or Twemproxy on top of Redis. In this case, the proxy does the distribution work, and from the adapter's side (Nebulex.Adapters.Redis), it would be only configuration. Instead of connecting the adapter against the Redis nodes, we connect it against the proxy nodes, this means, in the configuration, we set up the pool with the host and port pointing to the proxy.

Configuration options

In addition to Nebulex.Cache config options, the adapter supports the following options:

:mode - Defines the Redis configuration mode, which determines how the adapter connects to and distributes data across Redis instances.
- :standalone - Single Redis instance mode. Use this for simple setups or development environments. Creates a pool of connections to one Redis server. See the "Standalone" section for more options.
- :redis_cluster - Native Redis Cluster mode. Use this for production distributed caching with automatic sharding, replication, and failover provided by Redis Cluster. The adapter automatically discovers cluster topology and routes commands to the correct shard. See the "Redis Cluster" section for more options.
- :client_side_cluster - Client-side sharding mode. Use this when you need to distribute data across multiple independent Redis instances without Redis Cluster. The adapter uses consistent hashing to distribute keys. See the "Client-side Cluster" section for more options.
The default value is :standalone.
:pool_size (pos_integer/0) - The number of connections per Redis instance or shard.
- In :standalone mode: Total number of connections to the single Redis instance.
- In :redis_cluster mode: Number of connections per shard (master node).
- In :client_side_cluster mode: Number of connections per node (can be overridden per node in the node configuration).
Defaults to System.schedulers_online(), which matches the number of available CPU cores and provides good concurrency.
:serializer - Custom serializer module implementing the Nebulex.Adapters.Redis.Serializer behaviour.
See the "Custom Serializers" section in the module documentation for examples.
:serializer_opts (keyword/0) - Options passed to the serializer module's encode/decode functions. These options are forwarded to your custom serializer and can be used to configure serialization behavior. The default value is [].
- :encode_key (keyword/0) - Options passed to Nebulex.Adapters.Redis.Serializer.encode_key/2 when encoding cache keys before storing them in Redis. The default value is [].
- :encode_value (keyword/0) - Options passed to Nebulex.Adapters.Redis.Serializer.encode_value/2 when encoding cache values before storing them in Redis. The default value is [].
- :decode_key (keyword/0) - Options passed to Nebulex.Adapters.Redis.Serializer.decode_key/2 when decoding cache keys retrieved from Redis. The default value is [].
- :decode_value (keyword/0) - Options passed to Nebulex.Adapters.Redis.Serializer.decode_value/2 when decoding cache values retrieved from Redis. The default value is [].
:conn_opts (keyword/0) - Redis connection options for :standalone mode. These options are passed directly to Redix when establishing connections.
For cluster modes, connection options are specified within :redis_cluster or :client_side_cluster configuration.
See Redix.start_link/1 for the complete list of available options.
The default value is [host: "127.0.0.1", port: 6379].
:redis_cluster - Required only when :mode is set to :redis_cluster. A keyword list of options.
See "Redis Cluster options" section below.
:client_side_cluster - Required only when :mode is set to :client_side_cluster. A keyword list of options.
See "Client-side Cluster options" section below.

Redis Cluster options

The available options are:

:configuration_endpoints - Required. A keyword list of Redis Cluster nodes used to discover and fetch the cluster topology.
Each endpoint is identified by an atom key and configured with connection options (same format as :conn_opts). The adapter tries each endpoint in order until it successfully retrieves the cluster topology using "CLUSTER SHARDS" (Redis >= 7) or "CLUSTER SLOTS" (Redis < 7).
Providing multiple endpoints improves reliability; if one node is unavailable, the adapter will try the next one.
See "Redis Cluster" for configuration examples.
:override_master_host (boolean/0) - Determines whether to override master host addresses returned by Redis Cluster with the configuration endpoint's host.
By default (false), the adapter uses host addresses returned by "CLUSTER SHARDS" (Redis >= 7) or "CLUSTER SLOTS" (Redis < 7). Set to true when:
- Running Redis in Docker, where advertised addresses differ from actual connection addresses.
- Redis nodes are behind NAT or a load balancer.
- The cluster advertises internal IPs unreachable from your application.
When true, the adapter replaces cluster-advertised hosts with the host from the configuration endpoint that provided the topology.
The default value is false.
:keyslot (function of arity 2) - Custom function to compute the Redis Cluster hash slot for a given key.
The function receives two arguments:
1. key - The cache key (after serialization).
2. range - The total number of hash slots (typically 16384).
It should return an integer between 0 and range - 1.
The default implementation uses CRC16-XMODEM algorithm with support for hash tags (e.g., {user}:123 and {user}:456 map to the same slot). Only provide a custom function if you need different hash slot calculation logic.
The default value is &Nebulex.Adapters.Redis.Cluster.Keyslot.hash_slot/2.

Client-side Cluster options

The available options are:

:nodes - Required. A keyword list of independent Redis nodes that form the client-side cluster.
Each node is identified by an atom key and configured with connection options. The adapter uses consistent hashing to distribute cache keys across these nodes. Each node can optionally override the global :pool_size setting.
See "Client-side Cluster" for configuration examples.

Shared runtime options

Since the adapter runs on top of Redix, all commands accept their options (e.g.: :timeout, and :telemetry_metadata). See Redix docs for more information.

Redis Cluster runtime options

The following options are only for the :redis_cluster mode and apply to all commands:

:lock_retries - When the config manager is running and setting up the hash slot map, all Redis commands get blocked until the cluster is properly configured and the hash slot map is ready to use. This option defines the max retry attempts to acquire the lock before executing the command. Defaults to :infinity.

Query API

The queryable API is implemented using Redis KEYS command for pattern matching and operations like get_all, delete_all, and count_all.

Performance Warning

The KEYS command can cause performance issues in production environments because it blocks Redis while scanning all keys in the database. Consider the following:

Avoid using pattern queries (get_all("pattern*")) in production with large datasets.
Prefer explicit key lists (get_all(in: [key1, key2])) when possible.
Use stream/2 instead of get_all/2 for large result sets to reduce memory usage.

This adapter currently uses the KEYS command. A refactoring to use SCAN (the Redis-recommended approach for production) is planned for the next release.

Keep in mind the following limitations:

Only keys can be queried (not values or other attributes).
Only strings and predefined queries are allowed as query values.
Pattern queries scan the entire keyspace.

See "KEYS" command for pattern syntax details.

Examples

iex> MyApp.RedisCache.put_all(%{
...>   "firstname" => "Albert",
...>   "lastname" => "Einstein",
...>   "age" => 76
...> })
:ok

# returns key/value pairs by default
iex> MyApp.RedisCache.get_all!("**name**") |> Map.new()
%{"firstname" => "Albert", "lastname" => "Einstein"}

iex> MyApp.RedisCache.get_all!("**name**", select: :key)
["firstname", "lastname"]

iex> MyApp.RedisCache.get_all!("a??", select: :key)
["age"]

iex> MyApp.RedisCache.get_all!(select: :key)
["age", "firstname", "lastname"]

iex> MyApp.RedisCache.stream!("**name**", select: :key) |> Enum.to_list()
["firstname", "lastname"]

Deleting/counting keys

iex> MyApp.RedisCache.delete_all!(in: ["foo", "bar"])
2
iex> MyApp.RedisCache.count_all!(in: ["foo", "bar"])
2

Transactions

Nebulex Transaction API

The Nebulex.Adapter.Transaction behaviour (for multi-operation transactions via the transaction/3 callback) is not currently implemented in this adapter, but it is planned for future releases.

However, the adapter does use Redis transactions (MULTI/EXEC) internally to ensure atomicity for operations like take/2, put_all/2, and update_counter/3. These internal transactions are transparent to users and happen automatically when needed.

Using the adapter as a Redis client

Since the Redis adapter works on top of Redix and provides features like connection pools, "Redis Cluster", etc., it may also work as a Redis client. The Redis API is quite extensive, and there are many useful commands we may want to run, leveraging the Redis adapter features. Therefore, the adapter provides additional functions to do so.

`fetch_conn(opts \\ [])`

The function accepts the following options:

:name (atom/0) - The name of the cache (in case you are using dynamic caches), otherwise it is not required (defaults to the cache module name).
:key (term/0) - A cache key used to determine which Redis instance/shard to connect to in cluster modes.
Required for :redis_cluster and :client_side_cluster modes, where the adapter uses the key to:
- In :redis_cluster mode: Calculate the hash slot to find the correct shard.
- In :client_side_cluster mode: Use consistent hashing to select the appropriate node.
When executing custom Redis commands (e.g., list operations, sorted sets), provide a key to ensure all operations target the same Redis instance. Not required for :standalone mode.

Let's see some examples:

iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LPUSH", "mylist", "hello"])
1
iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LPUSH", "mylist", "world"])
2
iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LRANGE", "mylist", "0", "-1"])
["hello", "world"]

When working with :redis_cluster or :client_side_cluster modes the option :key is required:

iex> {:ok, conn} = MyCache.fetch_conn(key: "mylist")
iex> Redix.pipeline!(conn, [
...>   ["LPUSH", "mylist", "hello"],
...>   ["LPUSH", "mylist", "world"],
...>   ["LRANGE", "mylist", "0", "-1"]
...> ])
[1, 2, ["hello", "world"]]

Since these functions run on top of Redix, they also accept their options (e.g.: :timeout, and :telemetry_metadata). See Redix docs for more information.

Encoding/decoding functions

The following functions are available to encode/decode Elixir terms. It is useful whenever you want to work with Elixir terms in addition to strings or other specific Redis data types.

encode_key(name \\ __MODULE__, key) - Encodes an Elixir term into a string. The argument name is optional and should be used in case of dynamic caches (Defaults to the defined cache module).
encode_value(name \\ __MODULE__, value) - Same as encode_key but it is specific for encoding values, in case the encoding for keys and values are different.
decode_key(name \\ __MODULE__, key) - Decodes binary into an Elixir term. The argument name is optional and should be used in case of dynamic caches (Defaults to the defined cache module).
decode_value(name \\ __MODULE__, value) - Same as decode_key but it is specific for decoding values, in case the decoding for keys and values are different.

Let's see some examples:

iex> conn = MyCache.fetch_conn!()
iex> key = MyCache.encode_key({:key, "key"})
iex> value = MyCache.encode_value({:value, "value"})
iex> Redix.command!(conn, ["SET", key, value], timeout: 5000)
"OK"
iex> Redix.command!(conn, ["GET", key]) |> MyCache.decode_value()
{:value, "value"}

Custom Serializers

By default, the adapter uses Erlang's term format (:erlang.term_to_binary/2) for serialization, with strings being stored as-is. You can implement a custom serializer by creating a module that implements the Nebulex.Adapters.Redis.Serializer behaviour.

This is useful when you need:

JSON serialization for interoperability with other systems.
Custom compression algorithms.
Different encoding strategies for keys vs values.
Integration with external serialization libraries.

Example custom serializer using JSON:

defmodule MyApp.JSONSerializer do
  @behaviour Nebulex.Adapters.Redis.Serializer

  @impl true
  def encode_key(key, _opts), do: JSON.encode!(key)

  @impl true
  def encode_value(value, _opts), do: JSON.encode!(value)

  @impl true
  def decode_key(key, _opts), do: JSON.decode!(key)

  @impl true
  def decode_value(value, _opts), do: JSON.decode!(value)
end

Then configure your cache to use the custom serializer:

config :my_app, MyApp.RedisCache,
  serializer: MyApp.JSONSerializer

Adapter-specific telemetry events for the `:redis_cluster` mode

Aside from the recommended Telemetry events by Nebulex.Cache, this adapter exposes the following Telemetry events for the :redis_cluster mode:

telemetry_prefix ++ [:redis_cluster, :setup, :start] - This event is specific to the :redis_cluster mode. Before the configuration manager calls Redis to set up the cluster shards, this event should be invoked.
The :measurements map will include the following:
- :system_time - The current system time in native units from calling: System.system_time().
A Telemetry :metadata map including the following fields:
- :adapter_meta - The adapter metadata.
- :pid - The configuration manager PID.
telemetry_prefix ++ [:redis_cluster, :setup, :stop] - This event is specific to the :redis_cluster mode. After the configuration manager set up the cluster shards, this event should be invoked.
The :measurements map will include the following:
- :duration - The time spent configuring the cluster. The measurement is given in the :native time unit. You can read more about it in the docs for System.convert_time_unit/3.
A Telemetry :metadata map including the following fields:
- :adapter_meta - The adapter metadata.
- :pid - The configuration manager PID.
- :status - The cluster setup status. If the cluster was configured successfully, the status will be set to :ok, otherwise, will be set to :error.
- :reason - The status reason. When the status is :ok, the reason is :succeeded, otherwise, it is the error reason.
telemetry_prefix ++ [:redis_cluster, :setup, :exception] - This event is specific to the :redis_cluster mode. When an exception is raised while configuring the cluster, this event should be invoked.
The :measurements map will include the following:
- :duration - The time spent configuring the cluster. The measurement is given in the :native time unit. You can read more about it in the docs for System.convert_time_unit/3.
A Telemetry :metadata map including the following fields:
- :adapter_meta - The adapter metadata.
- :pid - The configuration manager PID.
- :kind - The type of the error: :error, :exit, or :throw.
- :reason - The reason of the error.
- :stacktrace - The stacktrace.

Nebulex.Adapters.Redis (Nebulex.Adapters.Redis v3.0.0-rc.2)

Standalone

Redis Cluster

Client-side Cluster

Redis Proxy Alternative

Redis Proxy

Configuration options

Redis Cluster options

Client-side Cluster options

Shared runtime options

Redis Cluster runtime options

Query API

Performance Warning

Examples

Deleting/counting keys

Transactions

Nebulex Transaction API

Using the adapter as a Redis client

fetch_conn(opts \\ [])

Encoding/decoding functions

Custom Serializers

Adapter-specific telemetry events for the :redis_cluster mode

`fetch_conn(opts \\ [])`

Adapter-specific telemetry events for the `:redis_cluster` mode