A library to serialize between maps and nested structs.

Turns a map into a nested struct according to hints given to the library. And vice versa turns any nested struct into a map.

It works with maps/structs of any shape and level of nesting. Highly configurable by implementing Nestru.Decoder and Nestru.Encoder protocols for structs.

Useful for translating map keys to struct's fields named differently. Or to specify default values missing in the map and required by struct.

The library's primary purpose is to serialize a map coming from a JSON payload or an Erlang term; at the same time, the map can be of any origin.

The input map can have atom or binary keys. The library takes the binary key first and then the same-named atom key if the binary key is missing while decoding the map. The library generates maps with atom keys during the struct encode operation.

Tour

Let's say we have an Order with Total that we want to decode from a map. First, let's derive Nestru.Decoder protocol and specify that field :total should hold a value of Total struct like the following:

defmodule Order do
  @derive {Nestru.Decoder, hint: %{total: Total}}
  defstruct [:id, :total]
end

defmodule Total do
  @derive Nestru.Decoder
  defstruct [:sum]
end

Now we decode the Order from the nested map like that:

map = %{
  "id" => "A548",
  "total" => %{"sum" => 500}
}

{:ok, model} = Nestru.decode_from_map(map, Order)

{:ok, %Order{id: "A548", total: %Total{sum: 500}}}

We get the order as the expected nested struct. Good!

Now we add the :items field to Order1 struct to hold a list of LineItems:

defmodule Order1 do
  @derive {Nestru.Decoder, hint: %{total: Total}}
  defstruct [:id, :items, :total]
end

defmodule LineItem do
  @derive Nestru.Decoder
  defstruct [:amount]
end

and we decode the Order1 from the nested map like that:

map = %{
  "id" => "A548",
  "items" => [%{"amount" => 150}, %{"amount" => 350}],
  "total" => %{"sum" => 500}
}

{:ok, model} = Nestru.decode_from_map(map, Order1)

{:ok, %Order1{id: "A548", items: [%{"amount" => 150}, %{"amount" => 350}], total: %Total{sum: 500}}}

The :items field value of the %Order1{} is still the list of maps and not structs 🤔 This is because Nestru has no clue what kind of struct these list items should be. So let's give a hint to Nestru on how to decode that field:

defmodule Order2 do
  @derive {Nestru.Decoder, hint: %{total: Total, items: [LineItem]}}

  defstruct [:id, :items, :total]
end

Let's decode again:

{:ok, model} = Nestru.decode_from_map(map, Order2)

{:ok,
 %Order2{
   id: "A548",
   items: [%LineItem{amount: 150}, %LineItem{amount: 350}],
   total: %Total{sum: 500}
 }}

Voilà, we have field values as nested structs 🎉

For the case when the list contains several structs of different types, please, see the Serializing type-dependent fields section below.

Error handling and path to the failed part of the map

Every implemented function of Nestru protocols can return {error, message} tuple in case of failure. When Nestru receives the error tuple, it stops conversion and bypasses the error to the caller.

defmodule Location do
  @derive {Nestru.Decoder, hint: %{street: Street}}
  defstruct [:street]
end

defmodule Street do
  @derive {Nestru.Decoder, hint: %{house: House}}
  defstruct [:house]
end

defmodule House do
  defstruct [:number]

  defimpl Nestru.Decoder do
    def from_map_hint(_value, _context, map) do
      if Nestru.has_key?(map, :number) do
        {:ok, %{}}
      else
        {:error, "Can't continue without house number."}
      end
    end
  end
end

So when we decode the following map missing the number value, we will get the error back:

map = %{
  "street" => %{
    "house" => %{
      "name" => "Party house"
    }
  }
}

{:error, error} = Nestru.decode_from_map(map, Location)

{:error,
 %{
   get_in_keys: [#Function<8.67001686/3 in Access.key!/1>, #Function<8.67001686/3 in Access.key!/1>],
   message: "Can't continue without house number.",
   path: ["street", "house"]
 }}

Nestru wraps the error message into a map and adds path and get_in_keys fields to it. The path values point to the failed part of the map which can be returned like the following:

get_in(map, error.get_in_keys)

%{"name" => "Party house"}

Maps with different key names

In some cases, the map's keys have slightly different names compared to the target's struct field names. Fields that should be decoded into the struct can be gathered by adopting Nestru.PreDecoder protocol like the following:

defmodule Quote do
  @derive [
    {Nestru.PreDecoder, translate: %{"cost_value" => :cost}},
    Nestru.Decoder
  ]

  defstruct [:cost]
end

When we decode the map, Nestru will put the value of the "cost_value" key for the :cost key into the map and then complete the decoding:

map = %{
  "cost_value" => 1280
}

Nestru.decode_from_map(map, Quote)

{:ok, %Quote{cost: 1280}}

For more sophisticated key mapping you can implement the gather_fields_from_map/3 function of Nestru.PreDecoder explicitly.

Serializing type-dependent fields

To convert a struct with a field that can have the value of multiple struct types into the map and back, the type of the field's value should be persisted. It's possible to do that like the following:

defmodule BookCollection do
  defstruct [:name, :items]

  defimpl Nestru.Encoder do
    def gather_fields_from_struct(struct, _context) do
      items_kinds =
        Enum.map(struct.items, fn %module{} ->
          module
          |> Module.split()
          |> Enum.join(".")
        end)

      {:ok, %{name: struct.name, items: struct.items, items_kinds: items_kinds}}
    end
  end

  defimpl Nestru.Decoder do
    def from_map_hint(_value, _context, map) do
      items_kinds =
        Enum.map(map.items_kinds, fn module_string ->
          module_string
          |> String.split(".")
          |> Module.safe_concat()
        end)

      {:ok, %{items: &Nestru.decode_from_list_of_maps(&1, items_kinds)}}
    end
  end
end

defmodule BookCollection.Book do
  @derive [Nestru.Encoder, Nestru.Decoder]
  defstruct [:title]
end

defmodule BookCollection.Magazine do
  @derive [Nestru.Encoder, Nestru.Decoder]
  defstruct [:issue]
end

Let's convert the nested struct into a map. The returned map gets extra items_kinds field with types information:

alias BookCollection.{Book, Magazine}

collection = %BookCollection{
  name: "Duke of Norfolk's archive",
  items: [
    %Book{title: "The Spell in the Chasm"},
    %Magazine{issue: "Strange Hunt"}
  ]
}

{:ok, map} = Nestru.encode_to_map(collection)

{:ok,
 %{
   items: [%{title: "The Spell in the Chasm"}, %{issue: "Strange Hunt"}],
   items_kinds: ["BookCollection.Book", "BookCollection.Magazine"],
   name: "Duke of Norfolk's archive"
 }}

And restoring of the original nested struct is as simple as that:

{:ok, collection} = Nestru.decode_from_map(map, BookCollection)

{:ok,
 %BookCollection{
   items: [
     %BookCollection.Book{title: "The Spell in the Chasm"},
     %BookCollection.Magazine{issue: "Strange Hunt"}
   ],
   name: "Duke of Norfolk's archive"
 }}

Use with other libraries

Jason

JSON maps decoded with Jason library are supported with both binary and atoms keys.

ex_json_schema

ex_json_schema library can be used before decoding the input map with the JSON schema. To make sure that the structure of the input map is correct.

ExJSONPath

ExJsonPath library allows querying maps (JSON objects) and lists (JSON arrays), using JSONPath expressions. The queries can be useful in Nestru.PreDecoder.gather_fields_from_map/3 function to assemble fields for decoding from a map having a very different shape from the target struct.

Domo

You can use the Domo library to validate the t() types of the nested struct values after decoding with Nestru.

Domo can validate a nested struct in one pass, ensuring that the struct's field values match its t() type and associated preconditions.