barenboim

v0.3.1

barenboim v0.3.1 Barenboim

Barenboim is prepared to tackle with data streaming dependencies in concurrent flows.

If your application works with a data streaming and your incoming events could have dependencies between them, the app can have problems when:

  • The Application which is sending the data is not sending the data in the right order.
  • Your Application is treating the data concurrently therefore the order is not ensured.

Configuration

Barenboim uses poolboy and you can configure it depending on your needs:

config :barenboim,
  pool_domain: :global,     # default :local
  pool_size: 20,            # default 10
  max_overflow: 3           # default 5

You can also configure a delay for a reminder notification. A reminder notification is sent in order to avoid corner cases (notification between the data access and the registration of a dependency). This time (milliseconds) should be defined depending on your data access function time (see next section).

config :barenboim,
  reminder_time: 50     # default 100

How to use it

Define the function that will retrieve the dependency data where dependency_ref is the id of your data and call Barenboim.get_data. You can also specify a time out in milliseconds.

fun = fn(dependency_ref) -> MyDataModule.get(dependency_ref) end
{:ok, data} = Barenboim.get_data(dependency_ref, fun)

Meanwhile, the flow that is processing a new event has to notify when the data is available for others:

Barenboim.notify({:reference, dependency_ref})

Or you can even attach the data:

Barenboim.notify({:data, dependency_ref, dependency_data})

Summary

Functions

get_data(dependency_ref, fun, time_to_live \\ nil)

This function will return the dependency data when is ready

notify(dependency)

It has to be called when the data dependency is ready to be used by its dependents

Functions

get_data(dependency_ref, fun, time_to_live \\ nil) 
get_data(any, (any -> any), integer | nil) ::
  {:ok, any} |
  {:timeout, any}

This function will return the dependency data when is ready.

  • dependency_ref is the reference of the dependency data

  • fun is the function that will get the dependency data. If you don’t need the data in that moment, only ensure that the data is ready, your fun function could only be a data checker fn(dependency_ref) -> MyDataModule.exist(dependency_ref) end. Barenboim will consider a ready data when executing fun it retrieves a value different than:

    • nil
    • false
    • [] empty list
    • {} empty tuple

    • %{} empty map

      If some of these values are valid for your application, use encapsulation in your function, example: {:ok, []}

  • if time_to_live is a valid integer, Barenboim will only wait for the data these milliseconds. If the value does not arrive before the time out, it will return {:timeout, data} where data will be the returned data of fun at that moment. If no time_to_live is specified, or a not valid one, Barenboim will wait until the event arrives.

fun = fn(dependency_ref) -> MyDataModule.get(dependency_ref) end
case Barenboim.get_data(dependency_ref, fun) do
  {:ok, data} -> process(data)
  {:timeout, empty_data} -> go_on()
end
notify(dependency) 
notify({:reference, any} | {:data, any, any}) :: any

It has to be called when the data dependency is ready to be used by its dependents.

The argument can be a tuple with the atom :reference and the reference of the dependency or a tuple with the atom :data, the reference of the dependency and the dependency data.