View Source Reliable Scheduled Jobs
A common variant of recursive jobs are "scheduled jobs", where the goal is for a
job to repeat indefinitely with a fixed amount of time between executions. The
part that makes it "reliable" is the guarantee that we'll keep retrying the
job's business logic when the job retries, but we'll only schedule the next
occurrence once. In order to achieve this guarantee we'll make use of the
perform
function to receive a complete Oban.Job
struct.
Time for illustrative example!
use-case-delivering-daily-digest-emails
Use Case: Delivering Daily Digest Emails
When a new user signs up to use our site we need to start sending them daily digest emails. We want to deliver the emails around the same time a user signed up, repeating every 24 hours. It is important that we don't spam them with duplicate emails, so we ensure that the next email is only scheduled on our first attempt.
defmodule MyApp.Workers.ScheduledWorker do
use Oban.Worker, queue: :scheduled, max_attempts: 10
alias MyApp.Mailer
@one_day 60 * 60 * 24
@impl true
def perform(%{args: %{"email" => email} = args, attempt: 1}) do
args
|> new(schedule_in: @one_day)
|> Oban.insert!()
Mailer.deliver_email(email)
end
def perform(%{args: %{"email" => email}}) do
Mailer.deliver_email(email)
end
end
You'll notice that the first perform/1
clause only matches a job struct on the
first attempt. When it matches, the first clause schedules the next iteration
immediately, before attempting to deliver the email. Any subsequent retries
fall through to the second perform/1
clause, which only attempts to deliver
the email again. Combined, the clauses get us close to at-most-once semantics
for scheduling, and at-least-once semantics for delivery.
more-flexible-than-cron-scheduling
More Flexible Than CRON Scheduling
Delivering around the same time using cron-style scheduling would need extra book-keeping to check when a user signed up, and then only deliver to those users that signed up within that window of time. The recursive scheduling approach is more accurate and entirely self contained—when and if the digest interval changes the scheduling will pick it up automatically once our code deploys.
An extensive discussion on the Oban issue tracker prompted this example along with the underlying feature that made it possible.
considerations-for-scheduling-jobs-in-the-very-near-future
Considerations for Scheduling Jobs in the Very-Near-Future
If you use the schedule_in
or scheduled_at
options with a value that will
resolve to the very-near-future, for example:
# 1 second from now
%{}
|> new(schedule_in: 1)
|> Oban.insert()
# 500 milliseconds from now
very_soon = DateTime.utc_now() |> DateTime.add(500, :millisecond)
%{}
|> new(scheduled_at: very_soon)
|> Oban.insert()
your workers may not be aware of/attempt to perform the job until the next tick
as specific by the Oban Stager :interval
option. By default this is
set to 1_000ms
.
Be aware: Configuring the :interval
option below the recommended default
can have a considerable impact on database performance! It is not advised to
lower this value and should only be done as a last resort after considering
other ways to achieve your desired outcome.