A library to validate values of nested structs with their type spec t() and associated precondition functions.

Description

The library adds constructor, validation, and reflection functions to a struct module. When called, constructor and validation functions guarantee the following:

• A struct or a group of nested structs conforms to their t() types.
• The struct's data consistently follows the business rules given by type-associated precondition functions registered via precond macro.

If the conditions described above are not met, the constructor and validation functions return an error.

The business rule expressed via the precondition function is type-associated, which affects all structs referencing the appropriate type and using Domo.

In terms of Domain Driven Design, types and associated precondition functions define the invariants relating structs to each other.

use Domo

When you use Domo, the Domo module will define the following functions:

• new and new! to create a valid struct
• ensure_type and ensure_type! to validate the existing struct
• reflection functions required_fields, typed_fields, and __t__

See the Callbacks section for more details.

Custom constructor function

Sometimes a default value for the struct's field must be generated during the construction. To reuse the new(!)/1 function's name and keep the generated value validated, instruct Domo to use another name for its constructor with gen_constructor_name option, like the following:

defmodule Foo do
  use Domo, skip_defaults: true, gen_constructor_name: :_new

  defstruct [:id, :token]

  @type id :: non_neg_integer()
  @type token :: String.t()
  precond token: &byte_size(&1) == 8

  @type t :: %__MODULE__{id: id(), token: token()}

  def new(id) do
    _new(id: id, token: random_string(8))
  end

  def new!(id) do
    _new!(id: id, token: random_string(8))
  end

  defp random_string(length),
    do: :crypto.strong_rand_bytes(length) |> Base.encode64() |> binary_part(0, length)
end

Foo.new!(15245)

%Foo{id: 15245, token: "e8K9wP0e"}

Compile-time and Run-time validations

At the project's compile-time, Domo performs the following checks:

• It automatically validates that the default values given with defstruct/1 conform to the struct's type and fulfil preconditions (can be disabled, see __using__/1 for more details). You can turn off this behaviour by specifying skip_defaults option; see Configuration section for details.

• It ensures that the struct using Domo built with new(!)/1 function to be a default argument for a function or a default value for a struct's field matches its type and preconditions.

At run-time, Domo validates structs matching their t() types.

Domo compiles TypeEnsurer module from struct's t() type to do all kinds of validations. There is a generated function with pattern matchings and guards for each struct's field. Constructor and validation functions of the struct delegate the work to the appropriate TypeEnsurer module.

After the compilation, the flow of control of the nested StructA validation can look like the following:

+--------------------+
| PurchaseOrder      |     +---------------------------+
|                    |     | PurchaseOrder.TypeEnsurer |
| new(!)/1 ----------|--_  |                           |
| ensure_type(!)/1 --|-----|-> ensure_field_type/1     |
+--------------------+     |   private functions       |
                           +----|----------------------+
                                |
+-----------------------+       |
| LineItem              |       |  +-------------------------+
|                       |       |  | LineItem.TypeEnsurer    |
| new(!)/1              |       |  |                         |
| ensure_type(!)/1      |       +--|-> ensure_field_type/1   |
+-----------------------+          |   private functions     |
                                   +-------------------------+

In interactive mode (iex / livebook) Domo generates TypeEnsurer module dynamically as the last step of struct's module definition.

In mix compile mode Domo generates all TypeEnsurer modules after elixir compiler finishes its job. The generated code can be found in _build/MIX_ENV/domo_generated_code folder. However, that is for information purposes only. The following compilation will overwrite all changes there.

Depending types tracking

Let's suppose a structure field's type depends on a type defined in another module. When the latter type or its precondition changes, Domo recompiles the former module automatically to update its TypeEnsurer to keep the type validation up to date.

Domo tracks type-depending modules and touches appropriate files during compilation.

That works for any number of intermediate modules between the module defining the struct's field and the module defining the field's final type.

Integration with Ecto

Ecto schema changeset can be automatically validated to conform to t() type and associated preconditions. Then the changeset function can be shortened like the following:

defmodule Customer do
  use Ecto.Schema
  use Domo, skip_defaults: true

  import Ecto.Changeset
  import Domo.Changeset

  schema "customers" do
    field :first_name, :string
    field :last_name, :string
    field :birth_date, :date

    timestamps()
  end

  @type t :: %__MODULE__{
          first_name: String.t(),
          last_name: String.t(),
          birth_date: Date.t()
        }

  def changeset(changeset, attrs) do
    changeset
    |> cast(attrs, __schema__(:fields))
    # Domo.Changeset defines validate_type/1 function.
    |> validate_type()
  end 
end

The Domo validation comes with the price of about 1.5x times slower than the equivalent Ecto's validate_N functions.

See detailed example is in the example_avialia project.

Integration with libraries generating t() type for a struct

Domo is compatible with most libraries that generate t() type for a struct and an Ecto schema, f.e. typed_struct and typed_ecto_schema respectfully. Just use Domo in the module, and that's it.

An example is in the example_typed_integrations project.

TypedStruct's submodule generation with :module option currently is not supported.

Installation

To use Domo in a project, add the following line to mix.exs dependencies:

{:domo, "~> 1.5"}

And the following line to the project's mix.exs file:

compilers: [:domo_compiler] ++ Mix.compilers()

To exclude the generated TypeEnsurer modules from mix test --coverage add the following line, that works since Elixir v1.13, to the project's mix.exs:

test_coverage: [ignore_modules: [~r/\.TypeEnsurer$/]]

To avoid mix format putting extra parentheses around precond/1 macro call, add the following import to the .formatter.exs:

[
  import_deps: [:domo]
]

Phoenix hot-reload

To enable Phoenix hot-reload for struct's type ensurers built by Domo, update the compilers in the mix.exs file like the following:

compilers: [:domo_compiler] ++ Mix.compilers() ++ [:domo_phoenix_hot_reload]

And add the following line to the endpoint's configuration in the config.exs file:

config :my_app, MyApp.Endpoint,
  reloadable_compilers: [:phoenix, :domo_compiler] ++ Mix.compilers() ++ [:domo_phoenix_hot_reload]

Umbrella application

Add the Domo dependency and compilers config as mentioned in the section above to the mix.exs file for each app using Domo.

You may add the same domo compiler config line to the app itself and to the root umbrella's mix.exs to enable recompile command to work correctly for iex -S mix run in the root.

If you have the Phoenix hot-reload configured for one of the web apps in umbrella then the :domo_phoenix_hot_reload compiler should be added to all dependency apps used by the given web one.

Configuration

The options listed below can be set globally in the configuration with config :domo, option: value. The value given with use Domo, option: value overrides the global setting.

• skip_defaults - if set to true, disables the validation of default values given with defstruct/1 to conform to the t() type at compile time. Default is false.

• gen_constructor_name - the name of the constructor function added to the module. The raising error function name is generated automatically from the given one by adding trailing !. Defaults are new and new! appropriately.

• unexpected_type_error_as_warning - if set to true, prints warning instead of throwing an error for field type mismatch in the raising functions. Default is false.

• remote_types_as_any - keyword list of type lists by modules that should be treated as any(). F.e. [{ExternalModule, [:t, :name]}, {OtherModule, :t}] Default is nil.

Run the Application.put_env(:domo, :verbose_in_iex, true) to enable verbose messages from domo in Interactive Elixir console.

Limitations

Parametrized types are not supported because it adds lots of complexity. Library returns {:type_not_found, :key} error for @type dict(key, value) :: [{key, value}] definition. Library returns error for type referencing parametrized type like @type field :: container(integer()).

Primitive types referencing themselves are not supported. Library returns an error for @type leaf :: leaf | nil definition. On the other hand, structs referencing themselves are supported. The library will build TypeEnsurer for the following definition @type t :: %__MODULE__{leaf: t | nil} and validate.

Performance 🐢

Compilation-time

Library affects the project's full recompilation time almost insignificantly.

The compilation times for the business application with 38 structs (8 fields each on average) having 158 modules in total are the following:

Mode       Average (by 3 measurements)  Deviation
No Domo    14.826s                      11.92
With Domo  15.711s                      7.34

Comparison: 
No Domo    14.826s 
With Domo  15.711s - 1.06x slower

Run-time

The library ensures the correctness of data types at run-time, which comes with the computation price.

One of the standard tasks is translating the map with string values received as a form into a validated nested struct.

For benchmark, we use Album struct having many Track structs.

When comparing Domo generated new!/1 constructor function and Ecto changeset using a set of validate_.../2 functions internally, both have equivalent execution times. However, in the former case, the memory consumption is about 25% bigger.

In the case of Ecto changeset that uses Domo.Changeset.validate_type/1 function the computation takes about 1.5x times longer.

One of the benchmark results is shown below. The benchmark project is /benchmark_ecto_domo folder. You can run it with mix benchmark command.

Operating System: macOS
CPU Information: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
Number of Available Cores: 8
Available memory: 16 GB
Elixir 1.11.0
Erlang 24.3.3

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 8 s
memory time: 2 s
parallel: 1
inputs: none specified
Estimated total run time: 36 s

Benchmarking Domo Album.new!/1...
Benchmarking Domo.Changeset validate_type/1...
Benchmarking Ecto.Changeset validate_.../1...

Name                                     ips        average  deviation         median         99th %
Domo Album.new!/1                       5.31      188.25 ms     ±1.79%      188.28 ms      196.57 ms
Ecto.Changeset validate_.../1           5.21      191.85 ms     ±1.94%      191.00 ms      202.22 ms
Domo.Changeset validate_type/1          3.76      266.19 ms     ±1.20%      266.58 ms      271.01 ms

Comparison: 
Domo Album.new!/1                       5.31
Ecto.Changeset validate_.../1           5.21 - 1.02x slower +3.59 ms
Domo.Changeset validate_type/1          3.76 - 1.41x slower +77.93 ms

Memory usage statistics:

Name                              Memory usage
Domo Album.new!/1                    245.73 MB
Ecto.Changeset validate_.../1        186.59 MB - 0.76x memory usage -59.13956 MB
Domo.Changeset validate_type/1       238.69 MB - 0.97x memory usage -7.04444 MB

**All measurements for memory usage were the same**

Migration

To migrate to a new version of Domo, please, clean and recompile the project with mix clean --deps && mix compile command.

Adoption

It's possible to adopt Domo library in the project having user-defined constructor functions named new/1 by refactoring to the Domo generated one. Here's how:

1. Add :domo dependency to the project, configure compilers as described in the installation section
2. Set the name of the Domo generated constructor function by adding config :domo, :gen_constructor_name, :_new option into the confix.exs file, to prevent conflict with user-defined constructor function name
3. Add use Domo to existing struct, f.e. FirstStruct
4. Change all struct building calls to be done with Domo generated function with the name set on step 3 f.e. FistStruct._new(%{...})
5. Repeat for each struct in the project
6. Remove original new/1 if it's not needed anymore and rename _new to new in the whole project
7. Remove config :domo, :gen_constructor_name configuration because Domo generates constructor wiht new name by default.