M25

A background job queueing and scheduling library for Gleam!

Getting Started

Install M25:

gleam add m25

Create a pog connection. You should generally start both pog and m25 in a supervised manner.

import m25

pub fn main() -> Nil {
  let conn_name = process.new_name("db_connection")

  let conn_child =
    pog.default_config(conn_name)
    |> pog.host("localhost")
    |> pog.database("my_database")
    |> pog.pool_size(15)
    |> pog.supervised

  // Create a connection that can be accessed by our queue handlers
  let conn = pog.named_connection(conn_name)

  let assert Ok(m25) =
    m25.new(conn)
    |> m25.add_queue(email_queue())
    |> result.try(m25.add_queue(_, sms_queue()))

  let m25_child = m25.supervised(m25, 1000)

  let assert Ok(started) =
    static_supervisor.new(static_supervisor.OneForOne)
    |> static_supervisor.add(conn_child)
    |> static_supervisor.add(m25_child)
    |> static_supervisor.start

  // Your queues will now be set up and jobs will start executing
  // in the background!
}

You can then enqueue jobs using the m25.enqueue function.

let job =
  m25.new_job(job_input)
  |> m25.schedule(at: future_time)
  |> m25.retry(max_attempts: 3, delay: option.Some(duration.seconds(3)))

let assert Ok(_) = m25.enqueue(conn, my_queue, job)

Further documentation can be found at https://hexdocs.pm/m25. You will also find a runnable example in dev/m25_dev.gleam. This will run against the Postgres database configured in the compose.yml file.

Delivery Semantics

M25 aims to deliver jobs at least once. If a job fails, either by returning an error, crashing, timing out, or failing to respond to heartbeats, M25 will mark the job as failed and retry according to the configured retry policy.

Jobs that are stuck in an executing state will be cleaned up after a configurable timeout, and can be retried.

Tables

This driver creates tables in the m25 schema to store jobs.

The current tables created by this driver are:

Migrations

You can run migrations against your database using the provided CLI:

gleam run -m m25 migrate --addr=<connection_string>

The required SQL statements will be printed to the console for use with other migration tools like dbmate. Alternatively, you can apply the migrations directly using the --apply flag.

Support for Cigogne is in the works.

Development

docker compose up -d  # Start the local Postgres instance
gleam dev             # Run the example
gleam test            # Run the tests

Why ‘M25’?

This is the M25.