NebulexRedisAdapter (NebulexRedisAdapter v2.0.0) View Source

Nebulex adapter for Redis. This adapter is implemented using Redix, a Redis driver for Elixir.

NebulexRedisAdapter provides three setup alternatives:

  • Standalone - The adapter establishes a pool of connections with a single Redis node. The :standalone is the default mode.

  • Redis Cluster - Redis Cluster is a built-in feature in Redis since version 3, and it may be the most convenient and recommendable way to set up Redis in a cluster and have a distributed cache storage out-of-box. This adapter provides the :redis_cluster mode to set up Redis Cluster from the client-side automatically and be able to use it transparently.

  • Built-in client-side cluster based on sharding - This adapter provides a simple client-side cluster implementation based on Sharding distribution model via :client_side_cluster mode.

Shared Options

In addition to Nebulex.Cache shared options, this adapters supports the following options:

  • :mode - Defines the mode Redis will be set up. It can be one of the next values: :standalone, :client_side_cluster, :redis_cluster. Defaults to :standalone.

  • :pool_size - Number of connections in the pool. Defaults to System.schedulers_online().

  • :conn_opts - Redis client options (Redix options in this case). For more information about connection options, see Redix docs.

TTL or Expiration Time

As is explained in Nebulex.Cache, most of the write-like functions support the :ttl option to define the expiration time, and it is defined in milliseconds. Despite Redis work with seconds, the conversion logic is handled by the adapter transparently, so when using a cache even with the Redis adapter, be sure you pass the :ttl option in milliseconds.

Data Types

This adapter only works with strings internally, which means the given Elixir terms are encoded to binaries before executing the Redis command. The encoding/decoding process is performed by the adapter under-the-hood, so it is completely transparent for the user.

NOTE: Support for other Redis Data Types is in the roadmap.

Standalone

We can define a cache to use Redis as follows:

defmodule MyApp.RedisCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: NebulexRedisAdapter
end

The configuration for the cache must be in your application environment, usually defined in your config/config.exs:

config :my_app, MyApp.RedisCache,
  conn_opts: [
    host: "127.0.0.1",
    port: 6379
  ]

Redis Cluster

We can define a cache to use Redis Cluster as follows:

defmodule MyApp.RedisClusterCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: NebulexRedisAdapter
end

The config:

config :my_app, MyApp.RedisClusterCache,
  mode: :redis_cluster,
  master_nodes: [
    [
      host: "127.0.0.1",
      port: 7000
    ],
    [
      url: "redis://127.0.0.1:7001"
    ],
    [
      url: "redis://127.0.0.1:7002"
    ]
  ],
  conn_opts: [
    # Redix options, except `:host` and `:port`; unless we have a cluster
    # of nodes with the same host and/or port, which doesn't make sense.
  ]

Redis Cluster Options

In addition to shared options, :redis_cluster mode supports the following options:

  • :master_nodes - The list with the configuration for the Redis cluster master nodes. The configuration for each master nodes contains the same options as :conn_opts. The adapter traverses the list trying to establish connection at least with one of them and get the cluster slots to finally setup the Redis cluster from client side properly. If one fails, the adapter retries with the next in the list, that's why at least one master node must be set.

  • :conn_opts - Same as shared options (optional). The :conn_opts will be applied to each connection pool with the cluster (they will override the host and port retrieved from cluster slots info). For that reason, be careful when setting :host or :port options since they will be used globally and can cause connection issues. Normally, we add here the desired client options except :host and :port. If you have a cluster with the same host for all nodes, in that case make sense to add also the :host option.

  • :pool_size - Same as shared options (optional). It applies to all cluster slots, meaning all connection pools will have the same size.

Client-side cluster

We can define a cache with "client-side cluster mode" as follows:

defmodule MyApp.ClusteredCache do
  use Nebulex.Cache,
    otp_app: :nebulex,
    adapter: NebulexRedisAdapter
end

The config:

config :my_app, MyApp.ClusteredCache,
  mode: :client_side_cluster,
  nodes: [
    node1: [
      pool_size: 10,
      conn_opts: [
        host: "127.0.0.1",
        port: 9001
      ]
    ],
    node2: [
      pool_size: 4,
      conn_opts: [
        url: "redis://127.0.0.1:9002"
      ]
    ],
    node3: [
      conn_opts: [
        host: "127.0.0.1",
        port: 9003
      ]
    ]
  ]

By default, the adapter uses NebulexRedisAdapter.ClientCluster.Keyslot for the keyslot. Besides, if :jchash is defined as dependency, the adapter will use consistent-hashing automatically. However, you can also provide your own implementation by implementing the Nebulex.Adapter.Keyslot and set it into the :keyslot option. For example:

defmodule MyApp.ClusteredCache.Keyslot do
  use Nebulex.Adapter.Keyslot

  @impl true
  def hash_slot(key, range) do
    # your implementation goes here
  end
end

And the config:

config :my_app, MyApp.ClusteredCache,
  mode: :client_side_cluster,
  keyslot: MyApp.ClusteredCache.Keyslot,
  nodes: [
    ...
  ]

Client-side cluster options

In addition to shared options, :client_side_cluster mode supports the following options:

  • :nodes - The list of nodes the adapter will setup the cluster with; a pool of connections is established per node. The :client_side_cluster mode enables resilience to be able to survive in case any node(s) gets unreachable. For each element of the list, we set the configuration for each node, such as :conn_opts, :pool_size, etc.

  • :keyslot - Defines the module implementing Nebulex.Adapter.Keyslot behaviour, used to compute the node where the command will be applied to. It is highly recommendable to provide a consistent hashing implementation.

Queryable API

Since the queryable API is implemented by using KEYS command:

  • Only strings (String.t()) are allowed as query parameter.
  • Only keys can be queried.

Examples

iex> MyApp.RedisCache.put_all(%{
...>   "firstname" => "Albert",
...>   "lastname" => "Einstein",
...>   "age" => 76
...> })
:ok

iex> MyApp.RedisCache.all("**name**")
["firstname", "lastname"]

iex> MyApp.RedisCache.all("a??")
["age"]

iex> MyApp.RedisCache.all()
["age", "firstname", "lastname"]

iex> stream = TestCache.stream("**name**")
iex> stream |> Enum.to_list()
["firstname", "lastname"]

# get the values for the returned queried keys
iex> "**name**" |> MyApp.RedisCache.all() |> MyApp.RedisCache.get_all()
%{"firstname" => "Albert", "lastname" => "Einstein"}

Using the cache for executing a Redis command or pipeline

Since NebulexRedisAdapter works on top of Redix and provides features like connection pools and "Redis Cluster" support, it may be seen also as a sort of Redis client, but it is meant to be used mainly with the Nebulex cache API. However, Redis API is quite extensive and there are a lot of useful commands we may want to run taking advantage of the NebulexRedisAdapter features. Therefore, the adapter injects two additional/extended functions to the defined cache: command!/3 and pipeline!/3.

command!(key \\ nil, name \\ __MODULE__, command)

iex> MyCache.command!("mylist", ["LPUSH", "mylist", "world"])
1
iex> MyCache.command!("mylist", ["LPUSH", "mylist", "hello"])
2
iex> MyCache.command!("mylist", ["LRANGE", "mylist", "0", "-1"])
["hello", "world"]

pipeline!(key \\ nil, name \\ __MODULE__, commands)

iex> cache.pipeline!("mylist", [
...>   ["LPUSH", "mylist", "world"],
...>   ["LPUSH", "mylist", "hello"],
...>   ["LRANGE", "mylist", "0", "-1"]
...> ])
[1, 2, ["hello", "world"]]

Arguments for command!/3 and pipeline!/3:

  • key - it is required when used the adapter in mode :redis_cluster or :client_side_cluster so that the node where the commands will take place can be selected properly. For :standalone it is optional.
  • name - The name of the cache in case you are using dynamic caches, otherwise it is not required.
  • commands - Redis commands.

Transactions

This adapter doesn't provide support for transactions, since there is no way to guarantee its execution on Redis itself, at least not in the way the Nebulex.Adapter.Transaction.transaction/3 works, because the anonymous function can have any kind of logic, which cannot be translated easily into Redis commands.

In the future, it is planned to add to Nebulex a multi-like function to perform multiple commands at once, perhaps that will be the best way to perform transactions via Redis.