ElasticsearchEx.Deserializer (Elasticsearch v1.9.1)

View Source

Utilities for converting Elasticsearch documents and responses into idiomatic Elixir data structures.

This module provides functions to transform Elasticsearch data—such as documents, sources, or hits—into Elixir types, handling type conversions based on provided mappings. It supports nested structures, lists, and streams, and allows for custom key transformation.

Features

  • Converts string-keyed Elasticsearch maps into Elixir maps, optionally transforming keys (e.g., to atoms).
  • Deserializes special Elasticsearch types:
    • "binary": Decodes base64-encoded strings.
    • "integer_range", "long_range": Converts range objects to Range.t().
    • "date_range": Converts date range objects to Date.Range.t() (for "strict_date" format).
    • "date": Converts date strings to Date.t() or DateTime.t() (for "strict_date" or "strict_date_time" formats).
  • Handles streams, lists, and single documents.
  • Gracefully falls back to original values for invalid or unrecognized formats.

Example Usage

# Basic document deserialization
iex(1)> doc = %{"_index" => "test", "_source" => %{"date" => "2023-01-01"}}
iex(2)> ElasticsearchEx.Deserializer.deserialize(doc, &Function.identity/1)
%{"_index" => "test", "_source" => %{"date" => ~D[2023-01-01]}}

# Stream deserialization
iex(1)> stream = Stream.map([doc], & &1)
iex(2)> Stream.run(ElasticsearchEx.Deserializer.deserialize(stream, &Function.identity/1))
[%{_index: "test", _source: %{date: ~D[2023-01-01]}}]

# Custom key function (e.g., to atoms)
iex> ElasticsearchEx.Deserializer.deserialize(doc, &String.to_atom/1)
%{_index: "test", _source: %{date: ~D[2023-01-01]}}

Summary

Types

mappings()

Elasticsearch mappings describing field types and formats. Should contain a "properties" key when provided as a map.

value()

Input data to be deserialized: a stream, list of documents, a single document, a document source, or a field value.

Functions

deserialize(value, key_mapper \\ &Function.identity/1)

Deserializes Elasticsearch documents, sources, or streams into Elixir data structures.

deserialize_field(value, mapping, key_mapper \\ &Function.identity/1)

Deserializes a field value according to its Elasticsearch mapping.

Types

mappings()

@type mappings() :: %{required(binary()) => any()}

Elasticsearch mappings describing field types and formats. Should contain a "properties" key when provided as a map.

value()

@type value() :: Enumerable.t() | %{required(binary()) => any()}

Input data to be deserialized: a stream, list of documents, a single document, a document source, or a field value.

Functions

deserialize(value, key_mapper \\ &Function.identity/1)

@spec deserialize(value(), (binary() -> any())) :: value()

Deserializes Elasticsearch documents, sources, or streams into Elixir data structures.

Accepts a stream, list, single document, or source map. Optionally, a key_mapper function can be provided to transform map keys (defaults to identity).

Returns:

  • A stream for stream input
  • A list for list input
  • A map for document/source input

Examples

# Document deserialization
iex(1)> doc = %{"_index" => "test", "_source" => %{"field" => "data"}}
iex(2)> ElasticsearchEx.Deserializer.deserialize(doc)
%{"_index" => "test", "_source" => %{"field" => "data"}}

# With key transformation
iex> ElasticsearchEx.Deserializer.deserialize(doc, &String.to_atom/1)
%{_index: "test", _source: %{field: "data"}}

# Graceful handling of invalid dates
iex(1)> doc = %{"_index" => "test", "_source" => %{"date" => "invalid"}}
iex(2)> ElasticsearchEx.Deserializer.deserialize(doc)
%{"_index" => "test", "_source" => %{"date" => "invalid"}}

deserialize_field(value, mapping, key_mapper \\ &Function.identity/1)

@spec deserialize_field(any(), mappings(), (binary() -> any())) :: any()

Deserializes a field value according to its Elasticsearch mapping.

Handles lists, nested maps (with "properties"), and scalar values. Supports special types such as "binary", "integer_range", "long_range", "date_range", and "date" (with format).

If the value or mapping does not match a supported type or format, the original value is returned.

Examples

# Binary field
iex> ElasticsearchEx.Deserializer.deserialize_field("SGVsbG8=", %{"type" => "binary"})
"Hello"

# Date range field
iex> ElasticsearchEx.Deserializer.deserialize_field(
...>   %{"gte" => "2023-01-01", "lte" => "2023-01-02"},
...>   %{"type" => "date_range", "format" => "strict_date"},
...> )
#Date.Range<...>