Dataloader

Dataloader provides an easy way efficiently load data in batches. It’s inspired by https://github.com/facebook/dataloader, although it makes some small API changes to better suit Elixir use cases.

Central to Dataloader is the idea of a source. A single Dataloader struct can have many different sources, which represent different ways to load data.

Here’s an example of a data loader using an ecto source, and then loading some organization data.

source = Dataloader.Ecto.new(MyApp.Repo)

# setup the loader
loader = Dataloader.new |> Dataloader.add_source(:db, source)

# load some things
loader =
  loader
  |> Dataloader.load(:db, Organization, 1)
  |> Dataloader.load_many(:db, Organization, [4, 9])

# actually retrieve them
loader = Dataloader.run(loader)

# Now we can get whatever values out we want
organizations = Dataloader.get_many(loader, :db, Organization, [1,4])

This will do a single SQL query to get all organizations by ids 1,4, and 9. You can load multiple batches from multiple sources, and then when run/1 is called batch will be loaded concurrently.

Here we named the source :db within our dataloader. More commonly though if you’re using Phoenix you’ll want to name it after one of your contexts, and have a different source used for each context. This provides an easy way to enforce data access rules within each context. See the DataLoader.Ecto moduledocs for more details

Options

There are two configuration options:

• timeout - The maximum timeout to wait for running a source, defaults to 1s more than the maximum timeout of all added sources. Set with care, timeouts should really only be set on sources.
• get_policy - This configures how the dataloader will behave when fetching data which may have errored when we tried to load it.

These can be set as part of the new/1 call. So, for example, to configure a dataloader that returns nil on error with a 5s timeout:

loader =
  Dataloader.new(
    get_policy: :return_nil_on_error,
    timeout: :timer.seconds(5)
  )

get_policy

There are three implemented behaviours for this:

• raise_on_error (default)- If successful, calling get/4 or get_many/4 will return the value. If there was an exception when trying to load any of the data, it will raise that exception
• return_nil_on_error - Behaves similar to raise_on_error but will just return nil instead of raising. It will still log errors
• tuples - This will return {:ok, value}/{:error, reason} tuples depending on a successful or failed load, allowing for more fine-grained error handling if required