Nestru (Nestru v0.3.2) View Source

A library to serialize between maps and nested structs.

Turns a map into a nested struct according to hints given to the library. And vice versa turns any nested struct into a map.

It works with maps/structs of any shape and level of nesting. Highly configurable by implementing Nestru.Decoder and Nestru.Encoder protocols for structs.

Useful for translating map keys to struct's fields named differently. Or to specify default values missing in the map and required by struct.

The library's primary purpose is to serialize a map coming from a JSON payload or an Erlang term; at the same time, the map can be of any origin.

The input map can have atom or binary keys. The library takes the binary key first and then the same-named atom key if the binary key is missing while decoding the map. The library generates maps with atom keys during the struct encode operation.

Tour

Let's say we have an Order with Total that we want to decode from a map. First, let's derive Nestru.Decoder protocol and specify that field :total should hold a value of Total struct like the following:

defmodule Order do
  @derive {Nestru.Decoder, hint: %{total: Total}}
  defstruct [:id, :total]
end

defmodule Total do
  @derive Nestru.Decoder
  defstruct [:sum]
end

Now we decode the Order from the nested map like that:

map = %{
  "id" => "A548",
  "total" => %{"sum" => 500}
}

{:ok, model} = Nestru.decode_from_map(map, Order)
{:ok, %Order{id: "A548", total: %Total{sum: 500}}}

We get the order as the expected nested struct. Good!

Now we add the :items field to Order1 struct to hold a list of LineItems:

defmodule Order1 do
  @derive {Nestru.Decoder, hint: %{total: Total}}
  defstruct [:id, :items, :total]
end

defmodule LineItem do
  @derive Nestru.Decoder
  defstruct [:amount]
end

and we decode the Order1 from the nested map like that:

map = %{
  "id" => "A548",
  "items" => [%{"amount" => 150}, %{"amount" => 350}],
  "total" => %{"sum" => 500}
}

{:ok, model} = Nestru.decode_from_map(map, Order1)
{:ok, %Order1{id: "A548", items: [%{"amount" => 150}, %{"amount" => 350}], total: %Total{sum: 500}}}

The :items field value of the %Order1{} is still the list of maps and not structs 🤔 This is because Nestru has no clue what kind of struct these list items should be. So let's give a hint to Nestru on how to decode that field:

defmodule Order2 do
  @derive {Nestru.Decoder, hint: %{total: Total, items: [LineItem]}}

  defstruct [:id, :items, :total]
end

Let's decode again:

{:ok, model} = Nestru.decode_from_map(map, Order2)
{:ok,
 %Order2{
   id: "A548",
   items: [%LineItem{amount: 150}, %LineItem{amount: 350}],
   total: %Total{sum: 500}
 }}

Voilà, we have field values as nested structs 🎉

For the case when the list contains several structs of different types, please, see the Serializing type-dependent fields section below.

Error handling and path to the failed part of the map

Every implemented function of Nestru protocols can return {error, message} tuple in case of failure. When Nestru receives the error tuple, it stops conversion and bypasses the error to the caller.

defmodule Location do
  @derive {Nestru.Decoder, hint: %{street: Street}}
  defstruct [:street]
end

defmodule Street do
  @derive {Nestru.Decoder, hint: %{house: House}}
  defstruct [:house]
end

defmodule House do
  defstruct [:number]

  defimpl Nestru.Decoder do
    def from_map_hint(_value, _context, map) do
      if Nestru.has_key?(map, :number) do
        {:ok, %{}}
      else
        {:error, "Can't continue without house number."}
      end
    end
  end
end

So when we decode the following map missing the number value, we will get the error back:

map = %{
  "street" => %{
    "house" => %{
      "name" => "Party house"
    }
  }
}

{:error, error} = Nestru.decode_from_map(map, Location)
{:error,
 %{
   get_in_keys: [#Function<8.67001686/3 in Access.key!/1>, #Function<8.67001686/3 in Access.key!/1>],
   message: "Can't continue without house number.",
   path: ["street", "house"]
 }}

Nestru wraps the error message into a map and adds path and get_in_keys fields to it. The path values point to the failed part of the map which can be returned like the following:

get_in(map, error.get_in_keys)
%{"name" => "Party house"}

Maps with different key names

In some cases, the map's keys have slightly different names compared to the target's struct field names. Fields that should be decoded into the struct can be gathered by adopting Nestru.PreDecoder protocol like the following:

defmodule Quote do
  @derive [
    {Nestru.PreDecoder, translate: %{"cost_value" => :cost}},
    Nestru.Decoder
  ]

  defstruct [:cost]
end

When we decode the map, Nestru will put the value of the "cost_value" key for the :cost key into the map and then complete the decoding:

map = %{
  "cost_value" => 1280
}

Nestru.decode_from_map(map, Quote)
{:ok, %Quote{cost: 1280}}

For more sophisticated key mapping you can implement the gather_fields_from_map/3 function of Nestru.PreDecoder explicitly.

Serializing type-dependent fields

To convert a struct with a field that can have the value of multiple struct types into the map and back, the type of the field's value should be persisted. It's possible to do that like the following:

defmodule BookCollection do
  defstruct [:name, :items]

  defimpl Nestru.Encoder do
    def gather_fields_from_struct(struct, _context) do
      items_kinds =
        Enum.map(struct.items, fn %module{} ->
          module
          |> Module.split()
          |> Enum.join(".")
        end)

      {:ok, %{name: struct.name, items: struct.items, items_kinds: items_kinds}}
    end
  end

  defimpl Nestru.Decoder do
    def from_map_hint(_value, _context, map) do
      items_kinds =
        Enum.map(map.items_kinds, fn module_string ->
          module_string
          |> String.split(".")
          |> Module.safe_concat()
        end)

      {:ok, %{items: &Nestru.decode_from_list_of_maps(&1, items_kinds)}}
    end
  end
end

defmodule BookCollection.Book do
  @derive [Nestru.Encoder, Nestru.Decoder]
  defstruct [:title]
end

defmodule BookCollection.Magazine do
  @derive [Nestru.Encoder, Nestru.Decoder]
  defstruct [:issue]
end

Let's convert the nested struct into a map. The returned map gets extra items_kinds field with types information:

alias BookCollection.{Book, Magazine}

collection = %BookCollection{
  name: "Duke of Norfolk's archive",
  items: [
    %Book{title: "The Spell in the Chasm"},
    %Magazine{issue: "Strange Hunt"}
  ]
}

{:ok, map} = Nestru.encode_to_map(collection)
{:ok,
 %{
   items: [%{title: "The Spell in the Chasm"}, %{issue: "Strange Hunt"}],
   items_kinds: ["BookCollection.Book", "BookCollection.Magazine"],
   name: "Duke of Norfolk's archive"
 }}

And restoring of the original nested struct is as simple as that:

{:ok, collection} = Nestru.decode_from_map(map, BookCollection)
{:ok,
 %BookCollection{
   items: [
     %BookCollection.Book{title: "The Spell in the Chasm"},
     %BookCollection.Magazine{issue: "Strange Hunt"}
   ],
   name: "Duke of Norfolk's archive"
 }}

Use with other libraries

Jason

JSON maps decoded with Jason library are supported with both binary and atoms keys.

ex_json_schema

ex_json_schema library can be used before decoding the input map with the JSON schema. To make sure that the structure of the input map is correct.

ExJSONPath

ExJsonPath library allows querying maps (JSON objects) and lists (JSON arrays), using JSONPath expressions. The queries can be useful in Nestru.PreDecoder.gather_fields_from_map/3 function to assemble fields for decoding from a map having a very different shape from the target struct.

Domo

You can use the Domo library to validate the t() types of the nested struct values after decoding with Nestru.

Domo can validate a nested struct in one pass, ensuring that the struct's field values match its t() type and associated preconditions.

Link to this section Summary

Functions

Decodes a list of maps into the list of the given struct values.

Similar to decode_from_list_of_maps/2 but checks if enforced struct's fields keys exist in the given maps.

Decodes a map into the given struct.

Similar to decode_from_map/3 but checks if enforced struct's fields keys exist in the given map.

Encodes the given list of structs into a list of maps.

Encodes the given struct into a map.

Gets the value for a specific key in map. Lookups a binary then an atom key.

Returns whether the given key exists in the given map as a binary or as an atom.

Link to this section Functions

Link to this function

decode_from_list_of_maps(list, struct_atoms, context \\ [])

View Source

Decodes a list of maps into the list of the given struct values.

The first argument is a list of maps.

If the second argument is a struct's module atom, then the function calls the decode_from_map/3 on each input list item.

If the second argument is a list of struct module atoms, the function calls the decode_from_map/3 function on each input list item with the module atom taken at the same index of the second list. In this case, both arguments should be of equal length.

The third argument is a context value to be passed to implemented functions of Nestru.PreDecoder and Nestru.Decoder protocols.

The function returns a list of structs or the first error from decode_from_map/3 function.

Link to this function

decode_from_list_of_maps!(list, struct_atoms, context \\ [])

View Source

Similar to decode_from_list_of_maps/2 but checks if enforced struct's fields keys exist in the given maps.

Returns a struct or raises an error.

Link to this function

decode_from_map(map, struct_module, context \\ [])

View Source

Decodes a map into the given struct.

The first argument is a map having key-value pairs. Supports both string and atom keys in the map.

The second argument is a struct's module atom.

The third argument is a context value to be passed to implemented functions of Nestru.PreDecoder and Nestru.Decoder protocols.

To give a hint on how to decode nested struct values or a list of such values for the given field, implement Nestru.Decoder protocol for the struct.

Function calls struct/2 to build the struct's value. Keys in the map that don't exist in the struct are automatically discarded.

Link to this function

decode_from_map!(map, struct_module, context \\ [])

View Source

Similar to decode_from_map/3 but checks if enforced struct's fields keys exist in the given map.

Returns a struct or raises an error.

Link to this function

encode_to_list_of_maps(list, context \\ nil)

View Source

Encodes the given list of structs into a list of maps.

Calls encode_to_map/2 for each struct in the list. The Nestru.Encoder protocol should be implemented for each struct module.

The function returns a list of maps or the first error from encode_to_map/2 function.

Link to this function

encode_to_list_of_maps!(list, context \\ nil)

View Source

Similar to encode_to_list_of_maps/2

Returns list of maps or raises an error.

Link to this function

encode_to_map(struct, context \\ nil)

View Source

Encodes the given struct into a map.

The first argument is a struct value to be encoded into map.

Encodes each field's value recursively when it is a struct or a list of structs.

The second argument is a context to be passed to Nestru.Encoder protocol function.

To insert additional fields or rename or drop existing ones before encoding into map, implement Nestru.Encoder protocol for the struct. That can be used to keep additional type information for the field that can have a value of various value types.

Link to this function

encode_to_map!(struct, context \\ nil)

View Source

Similar to encode_to_map/1.

Returns a map or raises an error.

Link to this function

get(map, key, default \\ nil)

View Source

Gets the value for a specific key in map. Lookups a binary then an atom key.

If key is present in map then its value value is returned. Otherwise, default is returned.

If default is not provided, nil is used.

Returns whether the given key exists in the given map as a binary or as an atom.