Validate a list of attrs maps (attrs_list) by passing them through an Ecto changeset, then upsert the valid items to the database that corresponds to a given Ecto repo_module (e.g. YourProject.Repo).

Using a changeset serves two purposes:

1. The changeset can be used to validate and transform the data.
2. Using a changeset allows this function to perform bulk upserts with nested associations.

For validation, each list item in the attrs_list is converted to a changeset for a given schema_module. By default, this function expects the schema module to contain a 2-arity function called :changeset. (See the #options section for more info.)

Basic example

iex> BulkUpsert.bulk_upsert(
...>   YourProject.Repo,
...>   YourProject.Persons.Person,
...>   _attrs_list = [
...>     %{id: 1, name: "Alice", age: 25, phone_number: "555-1234"},
...>     %{id: 2, name: "Bob", age: 35, phone_number: "555-2345"},
...>   ]
...> )
:ok

Options

• :changeset_function_atom - The name of the 2-arity changeset function to apply for the given schema_module (Default: :changeset)

• :insert_all_function_module - Instead of using the :insert_all function in the given repo_module, you may specify the name of a custom module to use instead. (Default: Inherited from the value specified in the repo_module function argument, e.g. YourProject.Repo))

  • Example: YourProject.OtherRepo

• :insert_all_function_atom - Instead of using your repo module's :insert_all function, you may pass a compatible equivalent that accepts the same arguments. (Default: :insert_all)

  • Example: :insert_all_with_autogenerated_timestamps

• :insert_all_opts - Pass custom opts to the insert_all/3 function. This option consists of a map whose key is the schema module that may have items being upserted, and the value is the YourProject.Repo.insert_all/3 opts that will be applied when items for that schema are being upserted. By default, this function is configured to replace all values in a given struct, except for the primary key(s) and the insert timestamp. (Default: %{})

  • Example: %{YourProject.Persons.Person => [on_conflict: {:nothing}]}

• :recover_changeset_errors - If the given fields in a changeset have errors, then replace them with a custom fallback value. (Default: %{})

  • Example: %{YourProject.Persons.Person => %{phone_number: "INVALID"}}

• :replace_all_except - If a row already exists, then all fields will be replaced except the primary key, and any fields specified here. (Default: [])

  • Example: [:field, :other_field]

• :timeout - The maximum timeout for a transaction. (Default: 15000)

  • Example: 60_000

Examples

Upsert a list of Person attrs using the changeset function YourProject.Persons.Person.upsert_changeset/2 to validate the attrs:

iex> attrs_list = [%{id: 1, name: "Alice", ...}]

iex> BulkUpsert.bulk_upsert(
...>   YourProject.Repo,
...>   YourProject.Persons.Person,
...>   attrs_list,
...>   changeset_function_atom: :upsert_changeset
...> )
:ok

Upsert a list of attrs, but overwrite the :name field if there is a conflict.

If using this option, you must declare each schema that will get a customized :insert_all_opts keyword list. Any schemas that are not given custom :insert_all_opts will overwrite all fields except the primary key:

iex> insert_all_opts = %{
...>   YourProject.Persons => [on_conflict: {:replace, [:name]}]
...> }

iex> BulkUpsert.bulk_upsert(
...>   YourProject.Repo,
...>   YourProject.Persons.Person,
...>   _attrs_list = [%{id: 1, name: "Alicia"}],
...>   insert_all_opts: insert_all_opts
...> )
:ok

Known limitations

• This function will not currently work with the :placeholders option of Ecto's insert_all/3 function. This is because the attrs are passed directly to the changesets for validation, so the placeholder values will not be parsed correctly. This functionality can be added later if needed.