View Source Babel (Babel v1.2.0)

CI Coverage Status Hexdocs.pm Hex.pm Hex.pm Downloads

Data transformations made easy.

Table of Contents

Installation

Simply add babel to your list of dependencies in your mix.exs:

def deps do
  [
    {:babel, "~> 1.0"}
  ]
end

Differences between the versions are explained in the Changelog.

Documentation gets generated with ExDoc and can be viewed at HexDocs.

Usage

Babel was born out of a desire to simplify non-trivial data transformation pipelines. To focus on the "happy path" instead of having to write a bunch of boilerplate error handling code.

But don't listen to me, take a look for yourself:

pipeline =
  Babel.begin()
  |> Babel.fetch(["some", "nested", "path"])
  |> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))

data = %{
  "some" => %{
    "nested" => %{
      "path" => [
        %{"string-key" => :value2},
        %{"string-key" => :value2},
        %{"string-key" => :value2}
      ]
    }
  }
}

Babel.apply(pipeline, data)
=> {:ok, [
   %{atom_key: :value1},
   %{atom_key: :value2},
   %{atom_key: :value3}
]}

Error Reporting

Since you'll most likely build non-trivial transformation pipelines with Babel - which can fail at any given step - Babel ships with elaborate error reporting:

pipeline =
  Babel.begin()
  |> Babel.fetch(["some", "nested", "path"])
  |> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))

data = %{
  "some" => %{
    "nested" => %{
      "path" => [
        %{"unexpected-key" => :value1},
        %{"unexpected-key" => :value2},
        %{"unexpected-key" => :value3}
      ]
    }
  }
}

Babel.apply!(pipeline, data)

Which will produce the following error:

** (Babel.Error) Failed to transform data: [not_found: "string-key", not_found: "string-key", not_found: "string-key"]

Root Cause(s):
1. Babel.Trace<ERROR>{
  data = %{"unexpected-key" => :value1}

  Babel.fetch("string-key")
  |=> {:error, {:not_found, "string-key"}}
}
2. Babel.Trace<ERROR>{
  data = %{"unexpected-key" => :value2}

  Babel.fetch("string-key")
  |=> {:error, {:not_found, "string-key"}}
}
3. Babel.Trace<ERROR>{
  data = %{"unexpected-key" => :value3}

  Babel.fetch("string-key")
  |=> {:error, {:not_found, "string-key"}}
}

Full Trace:
Babel.Trace<ERROR>{
  data = %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]}}}

  Babel.Pipeline<>
  |
  | Babel.fetch(["some", "nested", "path"])
  | |=< %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{...}, ...]}}}
  | |=> [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
  |
  | Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
  | |=< [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
  | |
  | | Babel.into(%{atom_key: Babel.fetch("string-key")})
  | | |=< %{"unexpected-key" => :value1}
  | | |
  | | | Babel.fetch("string-key")
  | | | |=< %{"unexpected-key" => :value1}
  | | | |=> {:error, {:not_found, "string-key"}}
  | | |
  | | |=> {:error, [not_found: "string-key"]}
  | |
  | | Babel.into(%{atom_key: Babel.fetch("string-key")})
  | | |=< %{"unexpected-key" => :value2}
  | | |
  | | | Babel.fetch("string-key")
  | | | |=< %{"unexpected-key" => :value2}
  | | | |=> {:error, {:not_found, "string-key"}}
  | | |
  | | |=> {:error, [not_found: "string-key"]}
  | |
  | | Babel.into(%{atom_key: Babel.fetch("string-key")})
  | | |=< %{"unexpected-key" => :value3}
  | | |
  | | | Babel.fetch("string-key")
  | | | |=< %{"unexpected-key" => :value3}
  | | | |=> {:error, {:not_found, "string-key"}}
  | | |
  | | |=> {:error, [not_found: "string-key"]}
  | |
  | |=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
  |
  |=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
}

Babel achieves this by keeping track of all applied steps in a Babel.Trace struct. Rendering of a Babel.Trace is done through a custom Inspect implementation.

You have to this information everywhere: in the Babel.Error message, in iex, and whenever you inspect a Babel.Error or Babel.Trace.

Contributing

Contributions are always welcome but please read our contribution guidelines before doing so.

Summary

Types

Arbitrary data structure that ought to be transformed.

Arbitrary term describing a Babel step or pipeline.

A term or list of terms describing a key you'd like to fetch from a data structure.

t()

Functions

Tries to transform the given data as described by the given Babel.Applicable.

Tries to transform the given data as described by the given Babel.Applicable.

Alias for fetch/1.

Returns true when the given value is a Babel.Pipeline or a built-in Babel.Step.

Begin a new (empty) Babel.Pipeline.

Equivalent to call(module, function_name, []).

Calls the specified function with the data as the first argument and the given list as additional arguments.

Casts the data to the a boolean, float, or integer.

Combines two steps into a Babel.Pipeline.

Always returns the given value, regardless of what data is passed in.

Always errors with the given reason. Useful in combination with match/1 or flat_map/1.

Fetches the given path from the data, erroring when it cannot be found.

Applies the Babel.Applicable returned by the given function to each element of an Enumerable, effectively allowing you to make a choice.

Equivalent to get(path, nil).

Fetches the given path from the data, returning the given default when it cannot be found.

Always returns the data it receives, effectively acting as a noop.

Transforms the received data into the given data structure, evaluating any Babel.Applicable it comes across.

Returns true when the given value is a Babel.Pipeline or a built-in Babel.Step.

Applies the given Babel.Applicable to each element of an Enumerable.

Applies the Babel.Applicable returned by the given function the data, effectively allowing you to make a choice.

Alias for identity/0.

Sets the on_error handler of a Babel.Pipeline which gets called with a Babel.Error when any given step of a pipeline fails.

Syntactic sugar for building a named Babel.Pipeline with an optional on_error handler.

Always returns the original data that was given to Babel.

Applies the given function to the data, basically "do whatever".

Like apply/2 but returns a Babel.Trace instead.

Like try/2 but returns the accumulated failure when all steps fail.

Returns the result of the first Babel.Applicable that succeeds.

Types

@type data() :: term()

Arbitrary data structure that ought to be transformed.

@type name() :: term()

Arbitrary term describing a Babel step or pipeline.

@type path() :: term() | [term()]

A term or list of terms describing a key you'd like to fetch from a data structure.

@type t() :: Babel.Applicable.t()
@type t(output) :: Babel.Applicable.t(output)
@type t(input, output) :: Babel.Applicable.t(input, output)

Functions

@spec apply(t(output), data()) :: {:ok, output} | {:error, Babel.Error.t()}
when output: any()

Tries to transform the given data as described by the given Babel.Applicable.

If the Babel.Applicable transforms the data successfully it returns {:ok, output}, where output is whatever the given Babel.Applicable produces.

In case of failure an {:error, Babel.Error.t} is returned, which contains the failure reason and a Babel.Trace that describes each transformation step. See Babel.Trace for details.

@spec apply!(t(output), data()) :: output | no_return() when output: any()

Tries to transform the given data as described by the given Babel.Applicable.

If the Babel.Applicable transforms the data successfully it returns output, where output is whatever the given Babel.Applicable produces.

In case of failure a Babel.Error.t is raised, whose message includes a Babel.Trace that describes each transformation step. See Babel.Trace for details.

@spec at(path()) :: t()

Alias for fetch/1.

@spec at(t(), path()) :: t()

Alias for fetch/2.

Returns true when the given value is a Babel.Pipeline or a built-in Babel.Step.

Examples

iex> Babel.babel?(Babel.identity())
true

iex> pipeline = :my_pipeline |> Babel.begin() |> Babel.fetch([:foo, :bar]) |> Babel.map(Babel.cast(:integer))
iex> Babel.babel?(pipeline)
true

iex> Babel.babel?(:something)
false

iex> Babel.babel?("different")
false
@spec begin(name()) :: Babel.Pipeline.t()

Begin a new (empty) Babel.Pipeline.

Link to this function

call(module, function_name)

View Source
@spec call(module(), function_name :: atom()) :: t()

Equivalent to call(module, function_name, []).

Link to this function

call(babel, module, function_name)

View Source
@spec call(t(), module(), function_name :: atom()) :: t()
@spec call(module(), function_name :: atom(), extra_args :: list()) :: t()

See call/4.

Link to this function

call(babel, module, function_name, extra_args)

View Source
@spec call(t(), module(), function_name :: atom(), extra_args :: list()) :: t()

Calls the specified function with the data as the first argument and the given list as additional arguments.

If you want to pass the data not as the first argument use then/1 instead.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.call(String, :trim, ["="])
iex> Babel.apply!(step, "= some string =")
" some string "

iex> pipeline = Babel.fetch("string") |> Babel.call(String, :trim, ["="])
iex> Babel.apply!(pipeline, %{"string" => "= some string ="})
" some string "
@spec cast(:boolean) :: t(boolean())
@spec cast(:integer) :: t(integer())
@spec cast(:float) :: t(float())

See cast/2.

@spec cast(t(), :boolean) :: t(boolean())
@spec cast(t(), :integer) :: t(integer())
@spec cast(t(), :float) :: t(float())

Casts the data to the a boolean, float, or integer.

To cast data to a different type use either call/2 or then/1.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.cast(:boolean)
iex> Babel.apply!(step, "true")
true
iex> Babel.apply!(step, "FALSE")
false
iex> Babel.apply!(step, " YeS  ")
true
iex> Babel.apply!(step, "   no")
false

iex> step = Babel.cast(:integer)
iex> Babel.apply!(step, "42")
42
iex> Babel.apply!(step, 42.6)
42
iex> Babel.apply!(step, "  42.6 ")
42

iex> step = Babel.cast(:float)
iex> Babel.apply!(step, "42")
42.0
iex> Babel.apply!(step, 42)
42.0
iex> Babel.apply!(step, "  42.6 ")
42.6

iex> pipeline = Babel.fetch("boolean") |> Babel.cast(:boolean)
iex> Babel.apply!(pipeline, %{"boolean" => " True "})
true
@spec chain(nil, next) :: next when next: t()
@spec chain(t(input, in_between), next :: t(in_between, output)) ::
  Babel.Pipeline.t(input, output)
when input: any(), in_between: any(), output: any()

Combines two steps into a Babel.Pipeline.

All steps in a pipeline are evaluated sequentially, an error stops the pipeline, unless an on_error handler has been set.

@spec choose((input -> t(input, output))) :: t(output)
when input: data(), output: term()

Alias for match/1.

@spec choose(t(), (input -> t(input, output))) :: t(output)
when input: data(), output: term()

Alias for match/2.

@spec const(value) :: t(value) when value: any()

Always returns the given value, regardless of what data is passed in.

Useful in combination with match/1 or flat_map/1.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.const(:my_cool_value)
iex> Babel.apply!(step, "does not matter")
:my_cool_value

iex> step = Babel.const(42)
iex> Babel.apply!(step, "does not matter")
42
Link to this function

fail(reason_or_function)

View Source
@spec fail(reason_or_function :: reason | (input -> reason)) :: t(no_return())
when input: any(), reason: any()

Always errors with the given reason. Useful in combination with match/1 or flat_map/1.

Examples

iex> step = Babel.fail(:my_cool_reason)
iex> {:error, babel_error} = Babel.apply(step, "does not matter")
iex> babel_error.reason
:my_cool_reason
@spec fetch(path()) :: t()

See fetch/2.

@spec fetch(t(), path()) :: t()

Fetches the given path from the data, erroring when it cannot be found.

Use get/2 to recover to a default.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.fetch(:some_key)
iex> Babel.apply!(step, %{some_key: "some value"})
"some value"

iex> step = Babel.fetch([:some_key, "nested key", 2])
iex> Babel.apply!(step, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:third

iex> pipeline = Babel.fetch(:some_key) |> Babel.fetch("nested key") |> Babel.fetch(-1)
iex> Babel.apply!(pipeline, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:fourth
@spec flat_map((input -> t(input, output))) :: t([output])
when input: data(), output: term()

See flat_map/2.

@spec flat_map(t(Enumerable.t(input)), (input -> t(input, output))) :: t([output])
when input: data(), output: term()

Applies the Babel.Applicable returned by the given function to each element of an Enumerable, effectively allowing you to make a choice.

Functionally equivalent to Babel.map(Babel.match(fn ... end)).

Use map/1 if you want to apply the same applicable to all.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.flat_map(fn
...>   map when is_map(map) -> Babel.fetch(:some_key)
...>   _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(step, [%{some_key: "some value"}, [not_a: "map"]])
["some value", :default_value]

iex> pipeline = Babel.fetch("list") |> Babel.flat_map(fn
...>   map when is_map(map) -> Babel.fetch(:some_key)
...>   _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(pipeline, %{"list" => [%{some_key: "some value"}, [not_a: "map"]]})
["some value", :default_value]
@spec get(path()) :: t()

Equivalent to get(path, nil).

@spec get(path(), default :: any()) :: t()

See get/3.

Link to this function

get(babel, path, default)

View Source
@spec get(t(), path(), default :: any()) :: t()

Fetches the given path from the data, returning the given default when it cannot be found.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.get(:some_key, :my_default)
iex> Babel.apply!(step, %{some_key: "some value"})
"some value"

iex> step = Babel.get(:some_key, :my_default)
iex> Babel.apply!(step, %{})
:my_default

iex> step = Babel.get([:some_key, "nested key", 2], :my_default)
iex> Babel.apply!(step, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:third

iex> pipeline = Babel.fetch([:some_key, "nested key"]) |> Babel.get(-1, :my_default)
iex> Babel.apply!(pipeline, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:fourth
@spec identity() :: t(input, input) when input: any()

Always returns the data it receives, effectively acting as a noop.

Useful in combination with match/1 or flat_map/1.

Examples

iex> step = Babel.identity()
iex> Babel.apply!(step, "some value")
"some value"

iex> step = Babel.identity()
iex> Babel.apply!(step, :another_value)
:another_value
@spec into(intoable) :: t(intoable) when intoable: Babel.Intoable.t()

See into/2.

@spec into(t(), intoable) :: t(intoable) when intoable: Babel.Intoable.t()

Transforms the received data into the given data structure, evaluating any Babel.Applicable it comes across.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.into(%{atom_key: Babel.fetch("string key")})
iex> Babel.apply!(step, %{"string key" => "some value"})
%{atom_key: "some value"}

iex> step = Babel.fetch(:map) |> Babel.into(%{atom_key: Babel.fetch("string key")})
iex> Babel.apply!(step, %{map: %{"string key" => "some value"}})
%{atom_key: "some value"}
Link to this macro

is_babel(babel)

View Source (macro)

Returns true when the given value is a Babel.Pipeline or a built-in Babel.Step.

Examples

iex> Babel.is_babel(Babel.identity())
true

iex> pipeline = :my_pipeline |> Babel.begin() |> Babel.fetch([:foo, :bar]) |> Babel.map(Babel.cast(:integer))
iex> Babel.is_babel(pipeline)
true

iex> Babel.is_babel(:something)
false

iex> Babel.is_babel("different")
false
@spec map(t(input, output)) :: t([output]) when input: data(), output: term()

See map/2.

@spec map(t(Enumerable.t(input)), t(input, output)) :: t([output])
when input: data(), output: term()

Applies the given Babel.Applicable to each element of an Enumerable.

Use flat_map/1 if you need to choose the applicable for each element.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.map(Babel.fetch(:some_key))
iex> Babel.apply!(step, [%{some_key: "value1"}, %{some_key: "value2"}])
["value1", "value2"]

iex> pipeline = Babel.fetch("list") |> Babel.map(Babel.fetch(:some_key))
iex> Babel.apply!(pipeline, %{"list" => [%{some_key: "value1"}, %{some_key: "value2"}]})
["value1", "value2"]
@spec match((input -> t(input, output))) :: t(output)
when input: data(), output: term()

See match/2.

@spec match(t(), (input -> t(input, output))) :: t(output)
when input: data(), output: term()

Applies the Babel.Applicable returned by the given function the data, effectively allowing you to make a choice.

Use then/1 if you just want to apply an arbitrary function.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.match(fn
...>   map when is_map(map) -> Babel.fetch(:some_key)
...>   _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(step, %{some_key: "some value"})
"some value"
iex> Babel.apply!(step, [not_a: "map"])
:default_value

iex> pipeline = Babel.fetch("nested") |> Babel.match(fn
...>   map when is_map(map) -> Babel.fetch(:some_key)
...>   _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(pipeline, %{"nested" => %{some_key: "some value"}})
"some value"
@spec noop() :: t(input, input) when input: any()

Alias for identity/0.

Link to this function

on_error(babel, function)

View Source
@spec on_error(t(), Babel.Pipeline.on_error(output)) :: t(output) when output: any()

Sets the on_error handler of a Babel.Pipeline which gets called with a Babel.Error when any given step of a pipeline fails.

Overwrites any previously set on_error handler.

Examples

iex> pipeline = Babel.fetch("some key") |> Babel.cast(:boolean) |> Babel.on_error(fn %Babel.Error{} -> :recover_to_ok_for_example end)
iex> Babel.apply!(pipeline, %{"some key" => "not a boolean"})
:recover_to_ok_for_example
Link to this macro

pipeline(name, list)

View Source (macro)

Syntactic sugar for building a named Babel.Pipeline with an optional on_error handler.

Note: For future versions we'd like to explore built-in pipeline caching,

  so that each pipeline only gets built _once_.

Examples

Without error handling

Babel.pipeline :my_pipeline do
  Babel.begin()
  |> Babel.fetch(["some", "path"])
  |> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))
end

Which would be equivalent to:

:my_pipeline
|> Babel.begin()
|> Babel.fetch(["some", "path"])
|> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))

With error handling

Babel.pipeline :my_pipeline do
  Babel.begin()
  |> Babel.fetch(["some", "path"])
  |> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))
else
  %Babel.Error{} = error ->
    # recover here in some way
end

Which would be equivalent to:

:my_pipeline
|> Babel.begin()
|> Babel.fetch(["some", "path"])
|> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))
|> Babel.on_error(fn %Babel.Error{} = error ->
    # recover here in some way
end)
@spec root() :: t()

Always returns the original data that was given to Babel.

Examples

iex> step = Babel.root()
iex> Babel.apply!(step, "some value")
"some value"

iex> pipeline = Babel.fetch(:list) |> Babel.map(Babel.into(%{
...>   nested_key: Babel.fetch(:key),
...>   root_key: Babel.root() |> Babel.fetch(:key)
...> }))
iex> Babel.apply!(pipeline, %{key: "root value", list: [%{key: "nested value1"}, %{key: "nested value2"}]})
[
  %{nested_key: "nested value1", root_key: "root value"},
  %{nested_key: "nested value2", root_key: "root value"}
]
@spec then((input -> Babel.Step.result_or_trace(output))) :: t(output)
when input: any(), output: any()

See then/3.

@spec then(t(input), (input -> Babel.Step.result_or_trace(output))) :: t(output)
when input: data(), output: term()
@spec then(name(), (input -> Babel.Step.result_or_trace(output))) :: t(output)
when input: any(), output: any()

See then/3.

Link to this function

then(babel, descriptive_name, function)

View Source
@spec then(t(input), name(), (input -> Babel.Step.result_or_trace(output))) ::
  t(output)
when input: data(), output: term()

Applies the given function to the data, basically "do whatever".

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.then(fn _ -> :haha_you_cant_stop_me_from_ignoring_the_input end)
iex> Babel.apply!(step, %{some_key: "some value"})
:haha_you_cant_stop_me_from_ignoring_the_input

iex> step = Babel.then(fn iso8601 ->
...>   with {:ok, datetime, _offset} <- DateTime.from_iso8601(iso8601) do
...>     {:ok, datetime}
...>   end
...> end)
iex> Babel.apply!(step, "2015-01-23T23:50:07Z")
~U[2015-01-23 23:50:07Z]

iex> pipeline = Babel.fetch("datetime") |> Babel.then(fn iso8601 ->
...>   with {:ok, datetime, _offset} <- DateTime.from_iso8601(iso8601) do
...>     {:ok, datetime}
...>   end
...> end)
iex> Babel.apply!(pipeline, %{"datetime" => "2015-01-23T23:50:07Z"})
~U[2015-01-23 23:50:07Z]
@spec trace(t(input, output), data()) :: Babel.Trace.t(input, output)
when input: any(), output: any()

Like apply/2 but returns a Babel.Trace instead.

@spec try(applicables :: [t(output), ...]) :: t(output) when output: any()

Like try/2 but returns the accumulated failure when all steps fail.

@spec try(t(input), applicables :: [t(input, output), ...]) :: t(input, output)
when input: any(), output: any()
@spec try(applicables :: t(output) | [t(output), ...], default) :: t(output | default)
when output: any(), default: any()

See try/3.

Link to this function

try(babel, applicables, default)

View Source
@spec try(
  t(input, output),
  applicables :: t(input, output) | [t(input, output), ...],
  default
) :: t(input, output | default)
when input: any(), output: any(), default: any()

Returns the result of the first Babel.Applicable that succeeds.

If none succeed it either returns the given default or an accumulated error.

Supports composition (you can pipe into this to create a Babel.Pipeline).

Examples

iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")])
iex> Babel.apply!(step, %{atom_key: "some value"})
"some value"

iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")])
iex> Babel.apply!(step, %{"string key" => "some value"})
"some value"

iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")])
iex> {:error, babel_error} = Babel.apply(step, %{})
iex> babel_error.reason
[{:not_found, :atom_key}, {:not_found, "string key"}]

iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")], :default_value)
iex> Babel.apply!(step, %{})
:default_value

iex> pipeline = Babel.fetch("map") |> Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")], :default_value)
iex> Babel.apply!(pipeline, %{"map" => %{}})
:default_value