Nebulex.Adapters.Redis (Nebulex.Adapters.Redis v3.0.0-rc.1)

Nebulex adapter for Redis. This adapter is implemented using Redix (a Redis driver for Elixir).

The adapter provides three setup alternatives:

• Standalone - The adapter establishes a pool of connections with a single Redis node. The :standalone is the default mode.

• Redis Cluster - Redis Cluster is a built-in feature in Redis since version 3, and it may be the most convenient and recommendable way to set up Redis in a cluster and have a distributed cache storage out-of-box. This adapter provides the :redis_cluster mode to set up Redis Cluster from the client-side automatically and be able to use it transparently.

• Built-in client-side cluster - The :client_side_cluster mode provides a simple client-side cluster implementation based on sharding distribution model.

Standalone

A cache that uses Redis is defined as follows:

defmodule MyApp.RedisCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: Nebulex.Adapters.Redis
end

The configuration for the cache must be in your application environment, usually defined in your config/config.exs:

config :my_app, MyApp.RedisCache,
  conn_opts: [
    host: "127.0.0.1",
    port: 6379
  ]

Redis Cluster

A cache that uses Redis Cluster can be defined as follows:

defmodule MyApp.RedisClusterCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: Nebulex.Adapters.Redis
end

As you may notices, nothing has changed, it is defined the same as the standalone mode. The change is in the configuration:

config :my_app, MyApp.RedisClusterCache,
  mode: :redis_cluster,
  redis_cluster: [
    configuration_endpoints: [
      endpoint1_conn_opts: [
        host: "127.0.0.1",
        port: 6379,
        # Add the password if 'requirepass' is on
        password: "password"
      ],
      ...
    ]
  ]

Client-side Cluster

Same as the previous modes, a cache is defined as:

defmodule MyApp.ClusteredCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: Nebulex.Adapters.Redis
end

The config:

config :my_app, MyApp.ClusteredCache,
  mode: :client_side_cluster,
  client_side_cluster: [
    nodes: [
      node1: [
        pool_size: 10,
        conn_opts: [
          host: "127.0.0.1",
          port: 9001
        ]
      ],
      node2: [
        pool_size: 4,
        conn_opts: [
          url: "redis://127.0.0.1:9002"
        ]
      ],
      node3: [
        conn_opts: [
          host: "127.0.0.1",
          port: 9003
        ]
      ],
      ...
    ]
  ]

Redis Proxy

Another option for "Redis Cluster" or the built-in "Client-side cluster" is using a proxy such as Envoy proxy or Twemproxy on top of Redis. In this case, the proxy does the distribution work, and from the adparter's side (Nebulex.Adapters.Redis), it would be only configuration. Instead of connect the adapter against the Redis nodes, we connect it against the proxy nodes, this means, in the config, we setup the pool with the host and port pointing to the proxy.

Configuration options

In addition to Nebulex.Cache config options, the adapter supports the following options:

• :mode - Redis configuration mode.

  • :standalone - A single Redis instance. See the "Standalone" section in the module documentation for more options.
  • :redis_cluster - Redis Cluster setup. See the "Redis Cluster" section in the module documentation for more options.
  • :client_side_cluster - See the "Client-side Cluster" section in the module documentation for more options.

  The default value is :standalone.

• :pool_size (pos_integer/0) - The number of connections that will be started by the adapter (based on the :mode). The default value is System.schedulers_online().

• :serializer - Custom serializer module implementing the Nebulex.Adapters.Redis.Serializer behaviour.

• :serializer_opts (keyword/0) - Custom serializer options. The default value is [].

  • :encode_key (keyword/0) - Options for encoding the key. The default value is [].

  • :encode_value (keyword/0) - Options for encoding the value. The default value is [].

  • :decode_key (keyword/0) - Options for decoding the key. The default value is [].

  • :decode_value (keyword/0) - Options for decoding the value. The default value is [].

• :conn_opts (keyword/0) - Redis client options. See Redix docs for more information about the options. The default value is [host: "127.0.0.1", port: 6379].

• :redis_cluster - Required only when :mode is set to :redis_cluster. A keyword list of options.

  See "Redis Cluster options" section below.

• :client_side_cluster - Required only when :mode is set to :client_side_cluster. A keyword list of options.

  See "Client-side Cluster options" section below.

Redis Cluster options

The available options are:

• :configuration_endpoints - Required. A keyword list of named endpoints where the key is an atom as an identifier and the value is another keyword list of options (same as :conn_opts).

  See "Redis Cluster" for more information.

• :override_master_host (boolean/0) - Defines whether the given master host should be overridden with the configuration endpoint or not. Defaults to false.

  The adapter uses the host returned by the "CLUSTER SHARDS" (Redis >= 7) or "CLUSTER SLOTS" (Redis < 7) command. One may consider set it to true for tests when using Docker for example, or when Redis nodes are behind a load balancer that Redis doesn't know the endpoint of. See Redis docs for more information.

  The default value is false.

• :keyslot (function of arity 2) - A function to compute the hash slot for a given key and range. The default value is &Nebulex.Adapters.Redis.Cluster.Keyslot.hash_slot/2.

Client-side Cluster options

The available options are:

• :nodes - Required. A keyword list of named nodes where the key is an atom as an identifier and the value is another keyword list of options (same as :conn_opts).

  See "Client-side Cluster" for more information.

Shared runtime options

Since the adapter runs on top of Redix, all commands accept their options (e.g.: :timeout, and :telemetry_metadata). See Redix docs for more information.

Redis Cluster runtime options

The following options are only for the :redis_cluster mode and apply to all commands:

• :lock_retries - When the config manager is running and setting up the hash slot map, all Redis commands get blocked until the cluster is properly configured and the hash slot map is ready to use. This option defines the max retry attempts to acquire the lock before executing the command. Defaults to :infinity.

Query API

Since the queryable API is implemented by using KEYS command, keep in mind the following caveats:

• Only keys can be queried.
• Only strings and predefined queries are allowed as query values.

See "KEYS" command.

Examples

iex> MyApp.RedisCache.put_all(%{
...>   "firstname" => "Albert",
...>   "lastname" => "Einstein",
...>   "age" => 76
...> })
:ok

# returns key/value pairs by default
iex> MyApp.RedisCache.get_all!("**name**") |> Map.new()
%{"firstname" => "Albert", "lastname" => "Einstein"}

iex> MyApp.RedisCache.get_all!("**name**", select: :key)
["firstname", "lastname"]

iex> MyApp.RedisCache.get_all!("a??", select: :key)
["age"]

iex> MyApp.RedisCache.get_all!(select: :key)
["age", "firstname", "lastname"]

iex> MyApp.RedisCache.stream!("**name**", select: :key) |> Enum.to_list()
["firstname", "lastname"]

Deleting/counting keys

iex> MyApp.RedisCache.delete_all!({:in, ["foo", "bar"]})
2
iex> MyApp.RedisCache.count_all!({:in, ["foo", "bar"]})
2

Transactions

Using the adapter as a Redis client

Since the Redis adapter works on top of Redix and provides features like connection pools, "Redis Cluster", etc., it may also work as a Redis client. The Redis API is quite extensive, and there are many useful commands we may want to run, leveraging the Redis adapter features. Therefore, the adapter provides additional functions to do so.

fetch_conn(opts \\ [])

The function accepts the following options:

• :name (atom/0) - The name of the cache (in case you are using dynamic caches), otherwise it is not required (defaults to the cache module name).

• :key (term/0) - The key is used to compute the node where the command will be executed. It is only required for :redis_cluster and :client_side_cluster modes.

Let's see some examples:

iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LPUSH", "mylist", "hello"])
1
iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LPUSH", "mylist", "world"])
2
iex> MyCache.fetch_conn!()
...> |> Redix.command!(["LRANGE", "mylist", "0", "-1"])
["hello", "world"]

When working with :redis_cluster or :client_side_cluster modes the option :key is required:

iex> {:ok, conn} = MyCache.fetch_conn(key: "mylist")
iex> Redix.pipeline!([
...>   ["LPUSH", "mylist", "hello"],
...>   ["LPUSH", "mylist", "world"],
...>   ["LRANGE", "mylist", "0", "-1"]
...> ])
[1, 2, ["hello", "world"]]

Since these functions run on top of Redix, they also accept their options (e.g.: :timeout, and :telemetry_metadata). See Redix docs for more information.

Encoding/decoding functions

The following functions are available to encode/decode Elixir terms. It is useful whenever you want to work with Elixir terms in addition to strings or other specific Redis data types.

• encode_key(name \\ __MODULE__, key) - Encodes an Elixir term into a string. The argument name is optional and should be used in case of dynamic caches (Defaults to the defined cache module).
• encode_value(name \\ __MODULE__, value) - Same as encode_key but it is specific for encoding values, in case the encoding for keys and values are different.
• decode_key(name \\ __MODULE__, key) - Decodes binary into an Elixir term. The argument name is optional and should be used in case of dynamic caches (Defaults to the defined cache module).
• decode_value(name \\ __MODULE__, value) - Same as decode_key but it is specific for decoding values, in case the decoding for keys and values are different.

Let's see some examples:

iex> conn = MyCache.fetch_conn!()
iex> key = MyCache.encode_key({:key, "key"})
iex> value = MyCache.encode_value({:value, "value"})
iex> Redix.command!(conn, ["SET", key, value], timeout: 5000)
"OK"
iex> Redix.command!(conn, ["GET", key]) |> MyCache.decode_value()
{:value, "value"}

Adapter-specific telemetry events for the :redis_cluster mode

Aside from the recommended Telemetry events by Nebulex.Cache, this adapter exposes the following Telemetry events for the :redis_cluster mode:

• telemetry_prefix ++ [:redis_cluster, :setup, :start] - This event is specific to the :redis_cluster mode. Before the configuration manager calls Redis to set up the cluster shards, this event should be invoked.

  The :measurements map will include the following:

  • :system_time - The current system time in native units from calling: System.system_time().

  A Telemetry :metadata map including the following fields:

  • :adapter_meta - The adapter metadata.
  • :pid - The configuration manager PID.

• telemetry_prefix ++ [:redis_cluster, :setup, :stop] - This event is specific to the :redis_cluster mode. After the configuration manager set up the cluster shards, this event should be invoked.

  The :measurements map will include the following:

  • :duration - The time spent configuring the cluster. The measurement is given in the :native time unit. You can read more about it in the docs for System.convert_time_unit/3.

  A Telemetry :metadata map including the following fields:

  • :adapter_meta - The adapter metadata.
  • :pid - The configuration manager PID.
  • :status - The cluster setup status. If the cluster was configured successfully, the status will be set to :ok, otherwise, will be set to :error.
  • :reason - The status reason. When the status is :ok, the reason is :succeeded, otherwise, it is the error reason.

• telemetry_prefix ++ [:redis_cluster, :setup, :exception] - This event is specific to the :redis_cluster mode. When an exception is raised while configuring the cluster, this event should be invoked.

  The :measurements map will include the following:

  • :duration - The time spent configuring the cluster. The measurement is given in the :native time unit. You can read more about it in the docs for System.convert_time_unit/3.

  A Telemetry :metadata map including the following fields:

  • :adapter_meta - The adapter metadata.
  • :pid - The configuration manager PID.
  • :kind - The type of the error: :error, :exit, or :throw.
  • :reason - The reason of the error.
  • :stacktrace - The stacktrace.