A behaviour module for implementing queues.

GenQueue relies on adapters to handle the specifics of how the queues are run. At its most simple, this can mean basic memory FIFO queues. At its most advanced, this can mean full async job queues with retries and backoffs. By providing a standard interface for such tools - ease in switching between different implementations is assured.

Example

The GenQueue behaviour abstracts the common queue interactions. Developers are only required to implement the callbacks and functionality they are interested in via adapters.

Let’s start with a simple FIFO queue:

defmodule Queue do
  use GenQueue
end

 # Start the queue
Queue.start_link()

# Push items into the queue
Queue.push(:hello)
#=> {:ok, :hello}
Queue.push(:world)
#=> {:ok, :world}

# Pop items from the queue
Queue.pop()
#=> {:ok, :hello}
Queue.pop()
#=> {:ok, :world}

We start our enqueuer by calling start_link/1. This call is then forwarded to our adapter. In this case, we dont specify an adapter anywhere, so it defaults to the simple FIFO queue implemented with the included GenQueue.Adapters.Simple.

We can then add items into our simple FIFO queues with push/2, as well as remove them with pop/1.

use GenQueue and adapters

As we can see from above - implementing a simple queue is easy. But we can further extend our queues by creating our own adapters or by using external libraries. Simply specify the adapter name in your config.

config :my_app, MyApp.Enqueuer, [
  adapter: MyApp.MyAdapter
]

defmodule MyApp.Enqueuer do
  use GenQueue, otp_app: :my_app
end

The adapter can also be specified for the module in line:

defmodule MyApp.Enqueuer do
  use GenQueue, adapter: MyApp.MyAdapter
end

We can then create our own adapter by creating an adapter module that handles the callbacks specified by GenQueue.Adapter.

defmodule MyApp.MyAdapter do
  use GenQueue.Adapter

  def handle_push(gen_queue, item) do
    IO.inspect(item)
    {:ok, item}
  end
end

Current adapters

Currently, the following adapters are available:

• GenQueue Exq - Redis-backed job queue.
• GenQueue TaskBunny - RabbitMQ-backed job queue.
• GenQueue Verk - Redis-backed job queue.
• GenQueue OPQ - GenStage-backed job queue.

Job queues

One of the benefits of using GenQueue is that it can abstract common tasks like job enqueueing. We can then provide a common API for the various forms of job enqueing we would like to implement, as well as easily swap implementations.

Please refer to the documentation for each adapter for more details.