View Source Schematic (schematic v0.1.1)

schematic is a library for data specification, validation, and transformation.

schematic works by constructing schematics that specify your data and can then unify to them from external data and dump your internal data back to the external data.

There are 12 builtin schematics that you can use to build new schematics that fit your own domain model.

Literals can be used as schematics and are unified with == semantics.

Struct literals can be used as schematics, and the input is unified by seeing if it is an instance of the given struct.

example

Example

Let's take a look at an example schematic for a JSON-RPC request for a bookstore API.

defmodule Bookstore do
  defmodule Datetime do
    import Schematic

    def schematic() do
      raw(
        fn
          i, :to -> is_binary(i) and match?({:ok, _, _}, DateTime.from_iso8601(i))
          i, :from -> match?(%DateTime{}, i)
        end,
        transform: fn
          i, :to ->
            {:ok, dt, _} = DateTime.from_iso8601(i)
            dt

          i, :from ->
            DateTime.to_iso8601(i)
        end
      )
    end
  end

  defmodule Author do
    import Schematic

    defstruct [:name]

    def schematic() do
      schema(__MODULE__, %{
        name: str()
      })
    end
  end

  defmodule Book do
    import Schematic

    defstruct [:title, :authors, :publication_date]

    def schematic() do
      schema(__MODULE__, %{
        {"publicationDate", :publication_date} => Bookstore.Datetime.schematic(),
        title: str(),
        authors: list(Bookstore.Author.schematic())
      })
    end
  end

  defmodule BooksListResult do
    import Schematic

    defstruct [:books]

    def schematic() do
      schema(__MODULE__, %{
        books: list(Bookstore.Book.schematic())
      })
    end
  end

  defmodule BooksListParams do
    import Schematic

    defstruct [:query, :order]

    def schematic() do
      schema(__MODULE__, %{
        query:
          nullable(
            map(%{
              {"field", :field} =>
                oneof(["title", "authors", "publication_date"]),
              {"value", :value} => str()
            })
          ),
        order: nullable(oneof(["asc", "desc"]))
      })
    end
  end

  defmodule BooksList do
    import Schematic

    defstruct [:id, :method, :params]

    def schematic() do
      schema(__MODULE__, %{
        id: int(),
        method: "books/list",
        params: Bookstore.BooksListParams.schematic()
      })
    end
  end
end

Reading external data into your data model.

iex> alias SchematicTest.Bookstore
iex> import Schematic
iex> unify(Bookstore.BooksList.schematic(), %{
...>   "id" => 99,
...>   "method" => "books/list",
...>   "params" => %{
...>     "query" => %{
...>       "field" => "authors",
...>       "value" => "Michael Crichton"
...>     },
...>     "order" => "desc"
...>   }
...> })
{:ok,
 %Bookstore.BooksList{
   id: 99,
   method: "books/list",
   params: %Bookstore.BooksListParams{
     query: %{field: "authors", value: "Michael Crichton"},
     order: "desc"
   }
 }}

Dumping your internal data model.

iex> alias SchematicTest.Bookstore
iex> import Schematic
iex> dump(Bookstore.BooksListResult.schematic(), %Bookstore.BooksListResult{
...>   books: [
...>     %Bookstore.Book{
...>       title: "Jurassic Park",
...>       authors: [%Bookstore.Author{name: "Michael Crichton"}],
...>       publication_date: ~U[1990-11-20 00:00:00.000000Z]
...>     },
...>     %Bookstore.Book{
...>       title: "The Lost World",
...>       authors: [%Bookstore.Author{name: "Michael Crichton"}],
...>       publication_date: ~U[1995-09-08 00:00:00.000000Z]
...>     }
...>   ]
...> })
{:ok,
%{
  "books" => [
    %{
      "authors" => [%{"name" => "Michael Crichton"}],
      "publicationDate" => "1990-11-20T00:00:00.000000Z",
      "title" => "Jurassic Park"
    },
    %{
      "authors" => [%{"name" => "Michael Crichton"}],
      "publicationDate" => "1995-09-08T00:00:00.000000Z",
      "title" => "The Lost World"
    }
  ]
}}

telemetry

Telemetry

schematic fires the following events:

  • [:schematic, :unify, :start] - Fired when unification starts.
  • [:schematic, :unify, :stop] - Fired when unification stops.
  • [:schematic, :unify, :exception] - Fired when unification raises an exception.

Link to this section Summary

Types

A lazy reference to a schematic, used to define recursive schematics.

A literal schematic.

The blueprint used to specify a map schematic.

Map blueprint key.

Map blueprint value.

The blueprint used to specify a schema schematic.

Schema blueprint key.

Schema blueprint value.

t()

The Schematic data structure.

Functions

Specifies that the data must unify with all of the given schematics.

Specifies that the data can be anything.

Specifies that the data is a boolean or a specific boolean.

Dump your internal data to their external data structures.

Specifies that the data is a float or specific float.

Specifies that the data is an integer or a specific integer.

Specifies that the data is a list of any size and contains anything.

Specifies that the data is a list whose items unify to the given schematic.

Specifies that the data is a map with the given keys (literal values) that unify to the provided blueprint.

Map schematics can be extended by using map/2.

Shortcut for specifiying that a schematic can be either null or the schematic.

Specifies that the data unifies to one of the given schematics.

See map/1 for examples and explanation.

A utility for creating custom schematics.

Specifies a map/1 schematic that is then hydrated into a struct.

Specifies that the data is a string or a specific string.

Specifies that the data is a tuple of the given length where each element unifies to the schematic in the same position.

Unify external data with your internal data structures.

Link to this section Types

@type lazy_schematic() :: {atom(), atom(), [any()]}

A lazy reference to a schematic, used to define recursive schematics.

@type literal() :: any()

A literal schematic.

@type map_blueprint() :: %{required(map_blueprint_key()) => map_blueprint_value()}

The blueprint used to specify a map schematic.

@type map_blueprint_key() :: Schematic.OptionalKey.t() | any()

Map blueprint key.

@type map_blueprint_value() :: t() | lazy_schematic() | literal()

Map blueprint value.

@type schema_blueprint() :: %{
  required(schema_blueprint_key()) => schema_blueprint_value()
}

The blueprint used to specify a schema schematic.

Link to this type

schema_blueprint_key()

View Source
@type schema_blueprint_key() :: Schematic.OptionalKey.t() | atom()

Schema blueprint key.

Link to this type

schema_blueprint_value()

View Source
@type schema_blueprint_value() :: t() | lazy_schematic() | literal()

Schema blueprint value.

@opaque t()

The Schematic data structure.

This data structure is meant to be opaque to the user, but you can create your own for super niche use cases. But backwards compatiblility of this data structure is not guaranteed.

Link to this section Functions

@spec all([t()]) :: t()

Specifies that the data must unify with all of the given schematics.

On error, returns a list of validation messages.

If a schematic raises an exception, it is caught and the error "is invalid" is returned.

iex> schematic = all([int(), raw(&Kernel.<(&1, 10), message: "must be less than 10"), raw(&(Kernel.rem(&1, 2) == 0), message: "must be divisible by 2")])
iex> {:ok, 8} = unify(schematic, 8)
iex> {:error, ["must be less than 10", "must be divisible by 2"]} = unify(schematic, 15)
iex> {:error, ["expected an integer", "must be less than 10", "is invalid"]} = unify(schematic, "15")
@spec any() :: t()

Specifies that the data can be anything.

usage

Usage

iex> schematic = any()
iex> {:ok, "hi!"} = unify(schematic, "hi!")
iex> {:ok, [:one, :two, :three]} = unify(schematic, [:one, :two, :three])
iex> {:ok, true} = unify(schematic, true)
@spec bool() :: t()

Specifies that the data is a boolean or a specific boolean.

usage

Usage

Any boolean.

iex> schematic = bool()
iex> {:ok, true} = unify(schematic, true)
iex> {:ok, false} = unify(schematic, false)
iex> {:error, "expected a boolean"} = unify(schematic, :boom)

A boolean literal.

iex> schematic = true
iex> {:ok, true} = unify(schematic, true)
iex> {:error, "expected true"} = unify(schematic, :boom)
@spec dump(t() | literal(), any()) :: any()

Dump your internal data to their external data structures.

See all the other functions for information on how to create schematics.

@spec float() :: t()

Specifies that the data is a float or specific float.

usage

Usage

Any float

iex> schematic = float()
iex> {:ok, 99.0} = unify(schematic, 99.0)
iex> {:error, "expected a float"} = unify(schematic, :boom)

A float literal.

iex> schematic = 99.0
iex> {:ok, 99.0} = unify(schematic,  99.0)
iex> {:error, ~s|expected 99.0|} = unify(schematic, :ninetynine)
@spec int() :: t()

Specifies that the data is an integer or a specific integer.

usage

Usage

Any integer.

iex> schematic = int()
iex> {:ok, 99} = unify(schematic, 99)
iex> {:error, "expected an integer"} = unify(schematic, :boom)

A integer literal.

iex> schematic = 99
iex> {:ok, 99} = unify(schematic,  99)
iex> {:error, ~s|expected 99|} = unify(schematic, :ninetynine)
@spec list() :: t()

Specifies that the data is a list of any size and contains anything.

usage

Usage

iex> schematic = list()
iex> {:ok, ["one", 2, :three]} = unify(schematic, ["one", 2, :three])
iex> {:error, "expected a list"} = unify(schematic, :hi)
@spec list(t() | lazy_schematic() | literal()) :: t()

Specifies that the data is a list whose items unify to the given schematic.

usage

Usage

iex> schematic = list(oneof([str(), int()]))
iex> {:ok, ["one", 2, "three"]} = unify(schematic, ["one", 2, "three"])
iex> {:error, "expected a list of either a string or an integer"} = unify(schematic, ["one", 2, :three])
Link to this function

map(blueprint_or_opts \\ [])

View Source
@spec map(%{required(map_blueprint_key()) => map_blueprint_value()} | Keyword.t()) ::
  t()

Specifies that the data is a map with the given keys (literal values) that unify to the provided blueprint.

Unification errors for keys are returned in a map with the key as the key and the value as the error.

  • Map schematics serve as a way to permit certain keys and discard all others.
  • Keys are non-nullable unless the value schematic is marked with nullable/1. This allows the value of the key to be nil as well as the key to be absent from the source data.
  • Keys are considered required unless tagged with optional/1. This allows the entire key to be absent from the source data. If the key is present, it must unify according to the given schematic.

basic-usage

Basic Usage

The most basic map schematic can look like the following.

iex> schematic = map(%{
...>   "league" => oneof(["NBA", "MLB", "NFL"]),
...> })
iex> # ignores the `"team"` key
iex> {:ok, %{"league" => "NBA"}} == unify(schematic, %{"league" => "NBA", "team" => "Chicago Bulls"})
true
iex> {:error,
...>   %{
...>     "league" =>
...>     ~s|expected either "NBA", "MLB", or "NFL"|
...>   }} = unify(schematic, %{"league" => "NHL"})

with-a-permissive-amp

With a permissive amp

If you want to only check that the data is a map, but not the shape, you can use map/0.

iex> schematic = map()
iex> {:ok, %{"league" => "NBA"}} = unify(schematic, %{"league" => "NBA"})

with-nullable-1

With nullable/1

Marking a key as nullable using nullable/1.

This means the value of the key can be nil as well as omitting the key entirely. The unified output will always contain the key.

iex> schematic = map(%{
...>   "title" => str(),
...>   "description" => nullable(str())
...> })
iex> {:ok, %{"title" => "Elixir 101", "description" => nil}} = unify(schematic, %{"title" => "Elixir 101", "description" => nil})
iex> {:ok, %{"title" => "Elixir 101", "description" => nil}} = unify(schematic, %{"title" => "Elixir 101"})
iex> {:ok, %{"title" => "Elixir 101", "description" => nil}} = dump(schematic, %{"title" => "Elixir 101"})

with-optional-1

With optional/1

Marking a key as optional using optional/1.

This means that you can omit the key from the input and that the unified output will not contain the key if it wasn't in the input.

If the key is provided, it must unify according to the given schematic.

Likewise, using dump/2 will also omit that key.

iex> schematic = map(%{
...>   "title" => str(),
...>   optional("description") => str()
...> })
iex> {:ok, %{"title" => "Elixir 101", "description" =>  "An amazing programming course."}} = unify(schematic, %{"title" => "Elixir 101", "description" => "An amazing programming course."})
iex> {:ok, %{"title" => "Elixir 101"}} = unify(schematic, %{"title" => "Elixir 101"})
iex> {:ok, %{"title" => "Elixir 101"}} = dump(schematic, %{"title" => "Elixir 101"})

with-keys-and-values

With :keys and :values

Instead of passing a blueprint, which specifies keys and values, you can pass a :keys and :values options which provide schematic that all keys and values in the input must unify to.

iex> schematic = map(keys: str(), values: oneof([str(), int()]))
iex> {:ok, %{"type" => "big", "quantity" => 99}} = unify(schematic, %{"type" => "big", "quantity" => 99})
iex> {:error, %{"quantity" => "expected either a string or an integer"}} = unify(schematic, %{"type" => "big", "quantity" => [99]})

transforming-keys

Transforming Keys

During unification, key transformation can be performed if it is specified in the schematic.

You can specify a key as a 2-tuple with the first element being the input key and the second element being the output key. When calling dump/2, the key will be turned from the output key back to the input key (and will also be revalidated).

This is useful for transforming string keys to atom keys as well as camelCase keys to snake_case keys.

Key transformation can also be used when declaring an optional key with optional/1.

iex> schematic = map(%{
...>   {"teamName", :team_name} => str()
...> })
iex> {:ok, %{team_name: "Chicago Bulls"}} = unify(schematic, %{"teamName" => "Chicago Bulls"})
iex> {:ok, %{"teamName" => "Chicago Bulls"}} = dump(schematic, %{team_name: "Chicago Bulls"})

recursive-schematics

Recursive Schematics

One can define schematics that specify keys whose values are themselves.

For this to be possible, recursive schematics must terminate some way. This can be achienved by specifying those keys as optional/1 or within a oneof/1 schematic.

Recursive schematics are specified as a MFA tuple, lazy_schematic/0.

iex> defmodule Tree do
...>   import Schematic
...>
...>   def schematic() do
...>     map(%{values: list(Tree.branch())})
...>   end
...>
...>   def branch() do
...>     map(%{
...>       values: list(oneof([Tree.leaf(), {__MODULE__, :branch, []}]))
...>     })
...>   end
...>
...>   def leaf() do
...>     map(%{
...>       value: str()
...>     })
...>   end
...> end
iex> input = %{
...>   type: "root",
...>   values: [
...>     %{
...>       type: "branch",
...>       values: [
...>         %{
...>           type: "leaf",
...>           value: "i'm a leaf"
...>         },
...>         %{
...>           type: "branch",
...>           values: [
...>             %{
...>               type: "leaf",
...>               value: "i'm another leaf"
...>             }
...>           ]
...>         }
...>       ]
...>     }
...>   ]
...> }
iex> unify(SchematicTest.Tree.schematic(), input)
{:ok, %{values: [%{values: [%{value: "i'm a leaf"}, %{values: [%{value: "i'm another leaf"}]}]}]}}
Link to this function

map(schematic, blueprint)

View Source
@spec map(t(), map()) :: t()

Map schematics can be extended by using map/2.

iex> player = map(%{name: str(), team: str()})
iex> baseball_player = map(player, %{home_runs: int()})
iex> unify(baseball_player, %{name: "Sammy Sosa", team: "Cubs", home_runs: 609, favorite_food: "Hot Dog"})
{:ok, %{name: "Sammy Sosa", team: "Cubs", home_runs: 609}}
@spec nullable(t() | lazy_schematic() | literal()) :: t()

Shortcut for specifiying that a schematic can be either null or the schematic.

usage

Usage

iex> schematic = nullable(str())
iex> {:ok, nil} = unify(schematic, nil)
iex> {:ok, "hi!"} = unify(schematic, "hi!")
iex> {:error, "expected either null or a string"} = unify(schematic, :boom)
@spec oneof([t() | lazy_schematic() | literal()] | (any() -> t() | literal())) :: t()

Specifies that the data unifies to one of the given schematics.

Can be called with a list of schematics or a function.

with-a-list

With a list

When called with a list of schematics, they will be traversed during unification and the first one to unify will be returned. If none of them unify, then an error is returned.

iex> team = map(%{name: str(), league: str()})
iex> player = map(%{name: str(), team: str()})
iex> schematic = oneof([team, player])
iex> {:ok, %{name: "Indiana Pacers", league: "NBA"}} = unify(schematic, %{name: "Indiana Pacers", league: "NBA"})
iex> {:ok, %{name: "George Hill", team: "Indiana Pacers"}} = unify(schematic, %{name: "George Hill", team: "Indiana Pacers"})
iex> {:error, "expected either a map or a map"} = unify(schematic, %{name: "NBA", sport: "basketball"})

with-a-function

With a function

When called with a function, the input is passed as the only parameter. This can be used to dispach to a specific schematic. This is a performance optimization, as you can dispatch to a specific schematic rather than traversing all of them.

iex> schematic = oneof(fn
...>   %{type: "team"} -> map(%{name: str(), league: str()})
...>   %{type: "player"} -> map(%{name: str(), team: str()})
...>   _ -> {:error, "expected either a player or a team"}
...> end)
iex> {:ok, %{name: "Indiana Pacers", league: "NBA"}} = unify(schematic, %{type: "team", name: "Indiana Pacers", league: "NBA"})
iex> {:ok, %{name: "George Hill", team: "Indiana Pacers"}} = unify(schematic, %{type: "player", name: "George Hill", team: "Indiana Pacers"})
iex> {:error, "expected either a player or a team"} = unify(schematic, %{name: "NBA", sport: "basketball"})
@spec optional(any()) :: Schematic.OptionalKey.t()

See map/1 for examples and explanation.

Link to this function

raw(function, opts \\ [])

View Source
@spec raw((any() -> boolean()) | (any(), :up | :down -> boolean()), [tuple()]) :: t()

A utility for creating custom schematics.

The raw/1 schematic is useful for creating schematics that unify the values of the inputs, rather than just the shape.

options

Options

  • :message - a custom error message. Defaults to "is invalid".
  • :transformer - a function that takes the input and the unification direction and must return the desired value. Defaults to fn input, _dir -> input end.

basic-usage

Basic Usage

iex> schematic = all([int(), raw(fn i -> i > 10 end, message: "must be greater than 10")])
iex> {:ok, 11} = unify(schematic, 11)
iex> {:error, ["must be greater than 10"]} = unify(schematic, 9)

advanced-usage

Advanced Usage

If your data requires different validations for unification and dumping, then you can pass a 2-arity function (instead of a 1-arity function) and the second parameter will be the direction.

This concept also applies to the :transform option.

iex> schematic =
...>   raw(
...>     fn
...>       n, :to -> is_list(n) and length(n) == 3
...>       n, :from -> is_tuple(n) and tuple_size(n) == 3
...>     end,
...>     message: "must be a tuple of size 3",
...>     transform: fn
...>       input, :to ->
...>         List.to_tuple(input)
...>       input, :from ->
...>         Tuple.to_list(input)
...>     end
...>   )
iex> {:ok, {"one", "two", 3}} = unify(schematic, ["one", "two", 3])
iex> {:error, "must be a tuple of size 3"} = unify(schematic, ["not", "big"])
iex> {:ok, ["one", "two", 3]} = dump(schematic, {"one", "two", 3})
@spec schema(atom(), schema_blueprint()) :: t()

Specifies a map/1 schematic that is then hydrated into a struct.

Works the same as the map/1 schematic, but will also automatically transform all keys from string keys to atom keys if a key conversion is not already specified.

Since this schematic hydrates a struct, it is also only capable of having atom keys in the output, whereas a normal map can have arbitrary terms as the key.

iex> schematic =
...>   schema(HTTPRequest, %{
...>     method: oneof(["POST", "PUT", "PATCH"]),
...>     body: str()
...>   })
iex> {:ok, %HTTPRequest{method: "POST", body: ~s|{"name": "Peter"}|}} = unify(schematic, %{"method" => "POST", "body" => ~s|{"name": "Peter"}|})
iex> {:ok, %{"method" => "POST", "body" => ~s|{"name": "Peter"}|}} = dump(schematic, %HTTPRequest{method: "POST", body: ~s|{"name": "Peter"}|})
@spec str() :: t()

Specifies that the data is a string or a specific string.

usage

Usage

Any string.

iex> schematic = str()
iex> {:ok, "hi!"} = unify(schematic, "hi!")
iex> {:error, "expected a string"} = unify(schematic, :boom)

A string literal.

iex> schematic = "I 💜 Elixir"
iex> {:ok, "I 💜 Elixir"} = unify(schematic,  "I 💜 Elixir")
iex> {:error, ~s|expected "I 💜 Elixir"|} = unify(schematic, "I love Ruby")
Link to this function

tuple(schematics, opts \\ [])

View Source
@spec tuple([t() | lazy_schematic() | literal()], Keyword.t()) :: t()

Specifies that the data is a tuple of the given length where each element unifies to the schematic in the same position.

usage

Usage

iex> schematic = tuple([str(), int()])
iex> {:ok, {"one", 2}} = unify(schematic, {"one", 2})
iex> {:error, "expected a tuple of {a string, an integer}"} = unify(schematic, {1, "two"})

options

Options

  • :from - Either :tuple or :list. Defaults to :tuple.
iex> schematic = tuple([str(), int()], from: :list)
iex> {:ok, {"one", 2}} = unify(schematic, ["one", 2])
iex> {:error, "expected a list of {a string, an integer}"} = unify(schematic, [1, "two"])
@spec unify(t() | literal(), any()) :: any()

Unify external data with your internal data structures.

See all the other functions for information on how to create schematics.