Settings View Source Changelog for Oban v2.12

🌟 Looking for changes to Web or Pro? Check the Oban.Pro Changelog or the Oban.Web Changelog. 🌟

Oban v2.12 Upgrade Guide

Oban v2.12 was dedicated to enriching the testing experience and expanding config, plugin, and queue validation across all environments.

testing-modes

Testing Modes

Testing modes bring a new, vastly improved, way to configure Oban for testing. The new testing option makes it explicit that Oban should operate in a restricted mode for the given environment.

Behind the scenes, the new testing modes rely on layers of validation within Oban's Config module. Now production configuration is validated automatically during test runs. Even though queues and plugins aren't started in the test environment, their configuration is still validated.

To switch, stop overriding plugins and queues and enable a testing mode in your test.exs config:

config :my_app, Oban, testing: :manual

Testing in :manual mode is identical to testing in older versions of Oban: jobs won't run automatically so you can use helpers like assert_enqueued and execute them manually with Oban.drain_queue/2.

An alternate :inline allows Oban to bypass all database interaction and run jobs immediately in the process that enqueued them.

config :my_app, Oban, testing: :inline

Finally, new testing guides cover test setup, unit testing workers, integration testing queues, and testing dynamic configuration.

global-peer-module

Global Peer Module

Oban v2.11 introduced centralized leadership via Postgres tables. However, Postgres based leadership isn't always a good fit. For example, an ephemeral leadership mechanism is preferred for integration testing.

In that case, you can make use of the new :global powered peer module for leadership:

config :my_app, Oban,
  peer: Oban.Peers.Global,
  ...

v2-12-1-2022-05-24

v2.12.1 — 2022-05-24

bug-fixes

Bug Fixes

• [BasicEngine] Never fetch jobs that have reached max attempts

  This adds a safeguard to the fetch_jobs function to prevent ever hitting the attempt <= max_attempts check constraint. Hitting the constraint causes the query to fail, which crashes the producer and starts an infinite loop of crashes. The previous commit should prevent this situation from occurring at the "staging" level, but to be absolutely safe this change prevents it at the "fetching" level too.

  There is a very minor performance hit from this change because the query can no longer run as an index only scan. For systems with a modest number of available jobs the performance impact is indistinguishable.

• [Plugins] Prevent unexpectedly modifying jobs selected by subqueries

  Most applications don't run at a serializable isolation level. That allows subqueries to run within a transaction without having the conditions rechecked—only predicates on UPDATE or DELETE are re-checked, not on subqueries. That allows a race condition where rows may be updated without another evaluation.

• [Repo] Set query_opts in Repo.transaction options to prevent logging begin and commit events in development loggers.

• [BasicEngine] Remove the ORDER BY clause from unique queries

  The previous ORDER BY id DESC significantly hurts unique query performance when there are a lot of potential jobs to check. The ordering was originally added to make test cases predictable and isn't important for the actual behaviour of the unique check.

v2-12-0-2022-04-19

v2.12.0 — 2022-04-19

enhancements

Enhancements

• [Oban] Replace queue, plugin, and peer test configuration with a single :testing option. Now configuring Oban for testing only requires one change, setting the test mode to either :inline or :manual.

  • :inline—jobs execute immediately within the calling process and without touching the database. This mode is simple and may not be suitable for apps with complex jobs.
  • :manual—jobs are inserted into the database where they can be verified and executed when desired. This mode is more advanced and trades simplicity for flexibility.

• [Testing] Add with_testing_mode/2 to temporarily change testing modes within the context of a function.

  Once the application starts in a particular testing mode it can't be changed. That's inconvenient if you're running in :inline mode and don't want a particular job to execute inline.

• [Config] Add validate/1 to aid in testing dynamic Oban configuration.

• [Config] Validate full plugin and queue options on init, without the need to start plugins or queues.

• [Peers.Global] Add an alternate :global powered peer module.

• [Plugin] A new Oban.Plugin behaviour formalizes starting and validating plugins. The behaviour is implemented by all plugins, and is the foundation of enhanced config validation.

• [Plugin] Emit [:oban, :plugin, :init] event on init from every plugin.

bug-fixes-1

Bug Fixes

• [Executor ] Skip timeout check with an unknown worker

  When the worker can't be resolved we don't need to check the timeout. Doing so prevents returning a helpful "unknown worker" message, and instead causes a function error for nil.timeout/1.

• [Testing] Include log and prefix in generated conf for perform_job.

  The opts, and subsequent conf, built for perform_job didn't include the prefix or log options. That prevented functions that depend on a job's conf within perform/1 from running with the correct options.

• [Drainer] Retain the currently configured engine while draining a queue.

• [Watchman] Skip pausing queues when shutdown is immediate. This prevents queue's from interacting with the database during short test runs.

For changes prior to v2.12 see the v2.11 docs.