View Source Babel (Babel v1.2.0)
Data transformations made easy.
Table of Contents
Installation
Simply add babel
to your list of dependencies in your mix.exs
:
def deps do
[
{:babel, "~> 1.0"}
]
end
Differences between the versions are explained in the Changelog.
Documentation gets generated with ExDoc and can be viewed at HexDocs.
Usage
Babel
was born out of a desire to simplify non-trivial data transformation pipelines.
To focus on the "happy path" instead of having to write a bunch of boilerplate error handling code.
But don't listen to me, take a look for yourself:
pipeline =
Babel.begin()
|> Babel.fetch(["some", "nested", "path"])
|> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
data = %{
"some" => %{
"nested" => %{
"path" => [
%{"string-key" => :value2},
%{"string-key" => :value2},
%{"string-key" => :value2}
]
}
}
}
Babel.apply(pipeline, data)
=> {:ok, [
%{atom_key: :value1},
%{atom_key: :value2},
%{atom_key: :value3}
]}
Error Reporting
Since you'll most likely build non-trivial transformation pipelines with Babel
- which can fail at any given step - Babel
ships with elaborate error reporting:
pipeline =
Babel.begin()
|> Babel.fetch(["some", "nested", "path"])
|> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
data = %{
"some" => %{
"nested" => %{
"path" => [
%{"unexpected-key" => :value1},
%{"unexpected-key" => :value2},
%{"unexpected-key" => :value3}
]
}
}
}
Babel.apply!(pipeline, data)
Which will produce the following error:
** (Babel.Error) Failed to transform data: [not_found: "string-key", not_found: "string-key", not_found: "string-key"]
Root Cause(s):
1. Babel.Trace<ERROR>{
data = %{"unexpected-key" => :value1}
Babel.fetch("string-key")
|=> {:error, {:not_found, "string-key"}}
}
2. Babel.Trace<ERROR>{
data = %{"unexpected-key" => :value2}
Babel.fetch("string-key")
|=> {:error, {:not_found, "string-key"}}
}
3. Babel.Trace<ERROR>{
data = %{"unexpected-key" => :value3}
Babel.fetch("string-key")
|=> {:error, {:not_found, "string-key"}}
}
Full Trace:
Babel.Trace<ERROR>{
data = %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]}}}
Babel.Pipeline<>
|
| Babel.fetch(["some", "nested", "path"])
| |=< %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{...}, ...]}}}
| |=> [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
|
| Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
| |=< [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
| |
| | Babel.into(%{atom_key: Babel.fetch("string-key")})
| | |=< %{"unexpected-key" => :value1}
| | |
| | | Babel.fetch("string-key")
| | | |=< %{"unexpected-key" => :value1}
| | | |=> {:error, {:not_found, "string-key"}}
| | |
| | |=> {:error, [not_found: "string-key"]}
| |
| | Babel.into(%{atom_key: Babel.fetch("string-key")})
| | |=< %{"unexpected-key" => :value2}
| | |
| | | Babel.fetch("string-key")
| | | |=< %{"unexpected-key" => :value2}
| | | |=> {:error, {:not_found, "string-key"}}
| | |
| | |=> {:error, [not_found: "string-key"]}
| |
| | Babel.into(%{atom_key: Babel.fetch("string-key")})
| | |=< %{"unexpected-key" => :value3}
| | |
| | | Babel.fetch("string-key")
| | | |=< %{"unexpected-key" => :value3}
| | | |=> {:error, {:not_found, "string-key"}}
| | |
| | |=> {:error, [not_found: "string-key"]}
| |
| |=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
|
|=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
}
Babel
achieves this by keeping track of all applied steps in a Babel.Trace
struct.
Rendering of a Babel.Trace
is done through a custom Inspect
implementation.
You have to this information everywhere: in the Babel.Error
message, in iex
, and whenever you inspect
a Babel.Error
or Babel.Trace
.
Contributing
Contributions are always welcome but please read our contribution guidelines before doing so.
Summary
Types
Arbitrary data structure that ought to be transformed.
Arbitrary term describing a Babel step or pipeline.
A term or list of terms describing a key you'd like to fetch from a data structure.
Functions
Tries to transform the given data
as described by the given Babel.Applicable
.
Tries to transform the given data
as described by the given Babel.Applicable
.
Alias for fetch/2
.
Returns true when the given value is a Babel.Pipeline
or a built-in Babel.Step
.
Begin a new (empty) Babel.Pipeline
.
Equivalent to call(module, function_name, [])
.
Calls the specified function with the data as the first argument and the given list as additional arguments.
See cast/2
.
Casts the data to the a boolean, float, or integer.
Combines two steps into a Babel.Pipeline
.
Alias for match/1
.
Alias for match/2
.
Always returns the given value, regardless of what data is passed in.
Always errors with the given reason. Useful in combination with match/1
or flat_map/1
.
See fetch/2
.
Fetches the given path from the data, erroring when it cannot be found.
Applies the Babel.Applicable
returned by the given function to each element of an Enumerable
,
effectively allowing you to make a choice.
Equivalent to get(path, nil)
.
See get/3
.
Fetches the given path from the data, returning the given default when it cannot be found.
Always returns the data it receives, effectively acting as a noop.
See into/2
.
Transforms the received data into the given data structure, evaluating any
Babel.Applicable
it comes across.
Returns true when the given value is a Babel.Pipeline
or a built-in Babel.Step
.
See map/2
.
Applies the given Babel.Applicable
to each element of an Enumerable
.
See match/2
.
Applies the Babel.Applicable
returned by the given function the data,
effectively allowing you to make a choice.
Alias for identity/0
.
Sets the on_error
handler of a Babel.Pipeline
which gets called with a
Babel.Error
when any given step of a pipeline fails.
Syntactic sugar for building a named Babel.Pipeline
with an optional on_error
handler.
See then/3
.
Applies the given function to the data, basically "do whatever".
Like apply/2
but returns a Babel.Trace
instead.
Like try/2
but returns the accumulated failure when all steps fail.
Returns the result of the first Babel.Applicable
that succeeds.
Types
@type data() :: term()
Arbitrary data structure that ought to be transformed.
@type name() :: term()
Arbitrary term describing a Babel step or pipeline.
A term or list of terms describing a key you'd like to fetch from a data structure.
@type t() :: Babel.Applicable.t()
@type t(output) :: Babel.Applicable.t(output)
@type t(input, output) :: Babel.Applicable.t(input, output)
Functions
@spec apply(t(output), data()) :: {:ok, output} | {:error, Babel.Error.t()} when output: any()
Tries to transform the given data
as described by the given Babel.Applicable
.
If the Babel.Applicable
transforms the data successfully it returns {:ok, output}
,
where output
is whatever the given Babel.Applicable
produces.
In case of failure an {:error, Babel.Error.t}
is returned, which contains the failure
reason
and a Babel.Trace
that describes each transformation step. See Babel.Trace
for details.
Tries to transform the given data
as described by the given Babel.Applicable
.
If the Babel.Applicable
transforms the data successfully it returns output
,
where output
is whatever the given Babel.Applicable
produces.
In case of failure a Babel.Error.t
is raised, whose message includes a Babel.Trace
that describes each transformation step. See Babel.Trace
for details.
Alias for fetch/1
.
Alias for fetch/2
.
Returns true when the given value is a Babel.Pipeline
or a built-in Babel.Step
.
Examples
iex> Babel.babel?(Babel.identity())
true
iex> pipeline = :my_pipeline |> Babel.begin() |> Babel.fetch([:foo, :bar]) |> Babel.map(Babel.cast(:integer))
iex> Babel.babel?(pipeline)
true
iex> Babel.babel?(:something)
false
iex> Babel.babel?("different")
false
@spec begin(name()) :: Babel.Pipeline.t()
Begin a new (empty) Babel.Pipeline
.
Equivalent to call(module, function_name, [])
.
@spec call(t(), module(), function_name :: atom()) :: t()
@spec call(module(), function_name :: atom(), extra_args :: list()) :: t()
See call/4
.
Calls the specified function with the data as the first argument and the given list as additional arguments.
If you want to pass the data not as the first argument use then/1
instead.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.call(String, :trim, ["="])
iex> Babel.apply!(step, "= some string =")
" some string "
iex> pipeline = Babel.fetch("string") |> Babel.call(String, :trim, ["="])
iex> Babel.apply!(pipeline, %{"string" => "= some string ="})
" some string "
@spec cast(:boolean) :: t(boolean())
@spec cast(:integer) :: t(integer())
@spec cast(:float) :: t(float())
See cast/2
.
@spec cast(t(), :boolean) :: t(boolean())
@spec cast(t(), :integer) :: t(integer())
@spec cast(t(), :float) :: t(float())
Casts the data to the a boolean, float, or integer.
To cast data to a different type use either call/2
or then/1
.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.cast(:boolean)
iex> Babel.apply!(step, "true")
true
iex> Babel.apply!(step, "FALSE")
false
iex> Babel.apply!(step, " YeS ")
true
iex> Babel.apply!(step, " no")
false
iex> step = Babel.cast(:integer)
iex> Babel.apply!(step, "42")
42
iex> Babel.apply!(step, 42.6)
42
iex> Babel.apply!(step, " 42.6 ")
42
iex> step = Babel.cast(:float)
iex> Babel.apply!(step, "42")
42.0
iex> Babel.apply!(step, 42)
42.0
iex> Babel.apply!(step, " 42.6 ")
42.6
iex> pipeline = Babel.fetch("boolean") |> Babel.cast(:boolean)
iex> Babel.apply!(pipeline, %{"boolean" => " True "})
true
@spec chain(nil, next) :: next when next: t()
@spec chain(t(input, in_between), next :: t(in_between, output)) :: Babel.Pipeline.t(input, output) when input: any(), in_between: any(), output: any()
Combines two steps into a Babel.Pipeline
.
All steps in a pipeline are evaluated sequentially, an error stops the pipeline,
unless an on_error
handler has been set.
Alias for match/1
.
Alias for match/2
.
Always returns the given value, regardless of what data is passed in.
Useful in combination with match/1
or flat_map/1
.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.const(:my_cool_value)
iex> Babel.apply!(step, "does not matter")
:my_cool_value
iex> step = Babel.const(42)
iex> Babel.apply!(step, "does not matter")
42
@spec fail(reason_or_function :: reason | (input -> reason)) :: t(no_return()) when input: any(), reason: any()
Always errors with the given reason. Useful in combination with match/1
or flat_map/1
.
Examples
iex> step = Babel.fail(:my_cool_reason)
iex> {:error, babel_error} = Babel.apply(step, "does not matter")
iex> babel_error.reason
:my_cool_reason
See fetch/2
.
Fetches the given path from the data, erroring when it cannot be found.
Use get/2
to recover to a default.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.fetch(:some_key)
iex> Babel.apply!(step, %{some_key: "some value"})
"some value"
iex> step = Babel.fetch([:some_key, "nested key", 2])
iex> Babel.apply!(step, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:third
iex> pipeline = Babel.fetch(:some_key) |> Babel.fetch("nested key") |> Babel.fetch(-1)
iex> Babel.apply!(pipeline, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:fourth
See flat_map/2
.
@spec flat_map(t(Enumerable.t(input)), (input -> t(input, output))) :: t([output]) when input: data(), output: term()
Applies the Babel.Applicable
returned by the given function to each element of an Enumerable
,
effectively allowing you to make a choice.
Functionally equivalent to Babel.map(Babel.match(fn ... end))
.
Use map/1
if you want to apply the same applicable to all.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.flat_map(fn
...> map when is_map(map) -> Babel.fetch(:some_key)
...> _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(step, [%{some_key: "some value"}, [not_a: "map"]])
["some value", :default_value]
iex> pipeline = Babel.fetch("list") |> Babel.flat_map(fn
...> map when is_map(map) -> Babel.fetch(:some_key)
...> _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(pipeline, %{"list" => [%{some_key: "some value"}, [not_a: "map"]]})
["some value", :default_value]
Equivalent to get(path, nil)
.
See get/3
.
Fetches the given path from the data, returning the given default when it cannot be found.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.get(:some_key, :my_default)
iex> Babel.apply!(step, %{some_key: "some value"})
"some value"
iex> step = Babel.get(:some_key, :my_default)
iex> Babel.apply!(step, %{})
:my_default
iex> step = Babel.get([:some_key, "nested key", 2], :my_default)
iex> Babel.apply!(step, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:third
iex> pipeline = Babel.fetch([:some_key, "nested key"]) |> Babel.get(-1, :my_default)
iex> Babel.apply!(pipeline, %{some_key: %{"nested key" => [:first, :second, :third, :fourth]}})
:fourth
Always returns the data it receives, effectively acting as a noop.
Useful in combination with match/1
or flat_map/1
.
Examples
iex> step = Babel.identity()
iex> Babel.apply!(step, "some value")
"some value"
iex> step = Babel.identity()
iex> Babel.apply!(step, :another_value)
:another_value
@spec into(intoable) :: t(intoable) when intoable: Babel.Intoable.t()
See into/2
.
@spec into(t(), intoable) :: t(intoable) when intoable: Babel.Intoable.t()
Transforms the received data into the given data structure, evaluating any
Babel.Applicable
it comes across.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.into(%{atom_key: Babel.fetch("string key")})
iex> Babel.apply!(step, %{"string key" => "some value"})
%{atom_key: "some value"}
iex> step = Babel.fetch(:map) |> Babel.into(%{atom_key: Babel.fetch("string key")})
iex> Babel.apply!(step, %{map: %{"string key" => "some value"}})
%{atom_key: "some value"}
Returns true when the given value is a Babel.Pipeline
or a built-in Babel.Step
.
Examples
iex> Babel.is_babel(Babel.identity())
true
iex> pipeline = :my_pipeline |> Babel.begin() |> Babel.fetch([:foo, :bar]) |> Babel.map(Babel.cast(:integer))
iex> Babel.is_babel(pipeline)
true
iex> Babel.is_babel(:something)
false
iex> Babel.is_babel("different")
false
See map/2
.
@spec map(t(Enumerable.t(input)), t(input, output)) :: t([output]) when input: data(), output: term()
Applies the given Babel.Applicable
to each element of an Enumerable
.
Use flat_map/1
if you need to choose the applicable for each element.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.map(Babel.fetch(:some_key))
iex> Babel.apply!(step, [%{some_key: "value1"}, %{some_key: "value2"}])
["value1", "value2"]
iex> pipeline = Babel.fetch("list") |> Babel.map(Babel.fetch(:some_key))
iex> Babel.apply!(pipeline, %{"list" => [%{some_key: "value1"}, %{some_key: "value2"}]})
["value1", "value2"]
See match/2
.
Applies the Babel.Applicable
returned by the given function the data,
effectively allowing you to make a choice.
Use then/1
if you just want to apply an arbitrary function.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.match(fn
...> map when is_map(map) -> Babel.fetch(:some_key)
...> _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(step, %{some_key: "some value"})
"some value"
iex> Babel.apply!(step, [not_a: "map"])
:default_value
iex> pipeline = Babel.fetch("nested") |> Babel.match(fn
...> map when is_map(map) -> Babel.fetch(:some_key)
...> _ -> Babel.const(:default_value)
...> end)
iex> Babel.apply!(pipeline, %{"nested" => %{some_key: "some value"}})
"some value"
Alias for identity/0
.
@spec on_error(t(), Babel.Pipeline.on_error(output)) :: t(output) when output: any()
Sets the on_error
handler of a Babel.Pipeline
which gets called with a
Babel.Error
when any given step of a pipeline fails.
Overwrites any previously set on_error
handler.
Examples
iex> pipeline = Babel.fetch("some key") |> Babel.cast(:boolean) |> Babel.on_error(fn %Babel.Error{} -> :recover_to_ok_for_example end)
iex> Babel.apply!(pipeline, %{"some key" => "not a boolean"})
:recover_to_ok_for_example
Syntactic sugar for building a named Babel.Pipeline
with an optional on_error
handler.
Note: For future versions we'd like to explore built-in pipeline caching,
so that each pipeline only gets built _once_.
Examples
Without error handling
Babel.pipeline :my_pipeline do
Babel.begin()
|> Babel.fetch(["some", "path"])
|> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))
end
Which would be equivalent to:
:my_pipeline
|> Babel.begin()
|> Babel.fetch(["some", "path"])
|> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))
With error handling
Babel.pipeline :my_pipeline do
Babel.begin()
|> Babel.fetch(["some", "path"])
|> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))
else
%Babel.Error{} = error ->
# recover here in some way
end
Which would be equivalent to:
:my_pipeline
|> Babel.begin()
|> Babel.fetch(["some", "path"])
|> Babel.map(Babel.into(%{some_map: Babel.fetch(:some_key)}))
|> Babel.on_error(fn %Babel.Error{} = error ->
# recover here in some way
end)
@spec root() :: t()
Always returns the original data that was given to Babel
.
Examples
iex> step = Babel.root()
iex> Babel.apply!(step, "some value")
"some value"
iex> pipeline = Babel.fetch(:list) |> Babel.map(Babel.into(%{
...> nested_key: Babel.fetch(:key),
...> root_key: Babel.root() |> Babel.fetch(:key)
...> }))
iex> Babel.apply!(pipeline, %{key: "root value", list: [%{key: "nested value1"}, %{key: "nested value2"}]})
[
%{nested_key: "nested value1", root_key: "root value"},
%{nested_key: "nested value2", root_key: "root value"}
]
@spec then((input -> Babel.Step.result_or_trace(output))) :: t(output) when input: any(), output: any()
See then/3
.
@spec then(t(input), (input -> Babel.Step.result_or_trace(output))) :: t(output) when input: data(), output: term()
@spec then(name(), (input -> Babel.Step.result_or_trace(output))) :: t(output) when input: any(), output: any()
See then/3
.
@spec then(t(input), name(), (input -> Babel.Step.result_or_trace(output))) :: t(output) when input: data(), output: term()
Applies the given function to the data, basically "do whatever".
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.then(fn _ -> :haha_you_cant_stop_me_from_ignoring_the_input end)
iex> Babel.apply!(step, %{some_key: "some value"})
:haha_you_cant_stop_me_from_ignoring_the_input
iex> step = Babel.then(fn iso8601 ->
...> with {:ok, datetime, _offset} <- DateTime.from_iso8601(iso8601) do
...> {:ok, datetime}
...> end
...> end)
iex> Babel.apply!(step, "2015-01-23T23:50:07Z")
~U[2015-01-23 23:50:07Z]
iex> pipeline = Babel.fetch("datetime") |> Babel.then(fn iso8601 ->
...> with {:ok, datetime, _offset} <- DateTime.from_iso8601(iso8601) do
...> {:ok, datetime}
...> end
...> end)
iex> Babel.apply!(pipeline, %{"datetime" => "2015-01-23T23:50:07Z"})
~U[2015-01-23 23:50:07Z]
@spec trace(t(input, output), data()) :: Babel.Trace.t(input, output) when input: any(), output: any()
Like apply/2
but returns a Babel.Trace
instead.
Like try/2
but returns the accumulated failure when all steps fail.
@spec try(t(input), applicables :: [t(input, output), ...]) :: t(input, output) when input: any(), output: any()
@spec try(applicables :: t(output) | [t(output), ...], default) :: t(output | default) when output: any(), default: any()
See try/3
.
@spec try( t(input, output), applicables :: t(input, output) | [t(input, output), ...], default ) :: t(input, output | default) when input: any(), output: any(), default: any()
Returns the result of the first Babel.Applicable
that succeeds.
If none succeed it either returns the given default or an accumulated error.
Supports composition (you can pipe into this to create a Babel.Pipeline
).
Examples
iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")])
iex> Babel.apply!(step, %{atom_key: "some value"})
"some value"
iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")])
iex> Babel.apply!(step, %{"string key" => "some value"})
"some value"
iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")])
iex> {:error, babel_error} = Babel.apply(step, %{})
iex> babel_error.reason
[{:not_found, :atom_key}, {:not_found, "string key"}]
iex> step = Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")], :default_value)
iex> Babel.apply!(step, %{})
:default_value
iex> pipeline = Babel.fetch("map") |> Babel.try([Babel.fetch(:atom_key), Babel.fetch("string key")], :default_value)
iex> Babel.apply!(pipeline, %{"map" => %{}})
:default_value