gleam/dynamic/decode
The Dynamic type is used to represent dynamically typed data. That is, data
that we don’t know the precise type of yet, so we need to introspect the data to
see if it is of the desired type before we can use it. Typically data like this
would come from user input or from untyped languages such as Erlang or JavaScript.
This module provides the Decoder type and associated functions, which provides
a type-safe and composable way to convert dynamic data into some desired type,
or into errors if the data doesn’t have the desired structure.
The Decoder type is generic and has 1 type parameter, which is the type that
it attempts to decode. A Decoder(String) can be used to decode strings, and a
Decoder(Option(Int)) can be used to decode Option(Int)s
Decoders work using runtime reflection and the data structures of the target platform. Differences between Erlang and JavaScript data structures may impact your decoders, so it is important to test your decoders on all supported platforms.
The decoding technique used by this module was inspired by Juraj Petráš’
Toy, Go’s encoding/json, and Elm’s
Json.Decode. Thank you to them!
Examples
Dynamic data may come from various sources and so many different syntaxes could be used to describe or construct them. In these examples a pseudocode syntax is used to describe the data.
Simple types
This module defines decoders for simple data types such as string,
int, float, bit_array, and bool.
// Data:
// "Hello, Joe!"
let result = decode.run(data, decode.string)
assert result == Ok("Hello, Joe!")
Lists
The list decoder decodes Lists. To use it you must construct it by
passing in another decoder into the list function, which is the decoder that
is to be used for the elements of the list, type checking both the list and its
elements.
// Data:
// [1, 2, 3, 4]
let result = decode.run(data, decode.list(decode.int))
assert result == Ok([1, 2, 3, 4])
On Erlang this decoder can decode from lists, and on JavaScript it can decode from lists as well as JavaScript arrays.
Options
The optional decoder is used to decode values that may or may not
be present. In other environment these might be called “nullable” values.
Like the list decoder, the optional decoder takes another decoder,
which is used to decode the value if it is present.
// Data:
// 12.45
let result = decode.run(data, decode.optional(decode.int))
assert result == Ok(option.Some(12.45))
// Data:
// null
let result = decode.run(data, decode.optional(decode.int))
assert result == Ok(option.None)
This decoder knows how to handle multiple different runtime representations of
absent values, including Nil, None, null, and undefined.
Dicts
The dict decoder decodes Dicts and contains two other decoders, one
for the keys, one for the values.
// Data:
// { "Lucy" -> 10, "Nubi" -> 20 }
let result = decode.run(data, decode.dict(decode.string, decode.int))
assert result == Ok(dict.from_list([
#("Lucy", 10),
#("Nubi", 20),
]))
Indexing objects
The at decoder can be used to decode a value that is nested within
key-value containers such as Gleam dicts, Erlang maps, or JavaScript objects.
// Data:
// { "one" -> { "two" -> 123 } }
let result = decode.run(data, decode.at(["one", "two"], decode.int))
assert result == Ok(123)
Indexing arrays
If you use ints as keys then the at decoder can be used to index into
array-like containers such as Gleam or Erlang tuples, or JavaScript arrays.
// Data:
// ["one", "two", "three"]
let result = decode.run(data, decode.at([1], decode.string))
assert result == Ok("two")
Records
Decoding records from dynamic data is more complex and requires combining a decoder for each field and a special constructor that builds your records with the decoded field values.
// Data:
// {
// "score" -> 180,
// "name" -> "Mel Smith",
// "is-admin" -> false,
// "enrolled" -> true,
// "colour" -> "Red",
// }
let decoder = {
use name <- decode.field("name", decode.string)
use score <- decode.field("score", decode.int)
use colour <- decode.field("colour", decode.string)
use enrolled <- decode.field("enrolled", decode.bool)
decode.success(Player(name:, score:, colour:, enrolled:))
}
let result = decode.run(data, decoder)
assert result == Ok(Player("Mel Smith", 180, "Red", True))
Enum variants
Imagine you have a custom type where all the variants do not contain any values.
pub type PocketMonsterType {
Fire
Water
Grass
Electric
}
You might choose to encode these variants as strings, "fire" for Fire,
"water" for Water, and so on. To decode them you’ll need to decode the dynamic
data as a string, but then you’ll need to decode it further still as not all
strings are valid values for the enum. This can be done with the then
function, which enables running a second decoder after the first one
succeeds.
let decoder = {
use decoded_string <- decode.then(decode.string)
case decoded_string {
// Return succeeding decoders for valid strings
"fire" -> decode.success(Fire)
"water" -> decode.success(Water)
"grass" -> decode.success(Grass)
"electric" -> decode.success(Electric)
// Return a failing decoder for any other strings
_ -> decode.failure(Fire, "PocketMonsterType")
}
}
let result = decode.run(dynamic.from("water"), decoder)
assert result == Ok(Water)
let result = decode.run(dynamic.from("wobble"), decoder)
assert result == Error([DecodeError("PocketMonsterType", "String", [])])
Record variants
Decoding type variants that contain other values is done by combining the techniques from the “enum variants” and “records” examples. Imagine you have this custom type that you want to decode:
pub type PocketMonsterPerson {
Trainer(name: String, badge_count: Int)
GymLeader(name: String, speciality: PocketMonsterType)
}
And you would like to be able to decode these from dynamic data like this:
{
"type" -> "trainer",
"name" -> "Ash",
"badge-count" -> 1,
}
{
"type" -> "gym-leader",
"name" -> "Misty",
"speciality" -> "water",
}
Notice how both documents have a "type" field, which is used to indicate which
variant the data is for.
First, define decoders for each of the variants:
let trainer_decoder = {
use name <- decode.field("name", decode.string)
use badge_count <- decode.field("badge-count", decode.int)
decode.success(Trainer(name, badge_count))
})
let gym_leader_decoder = {
use name <- decode.field("name", decode.string)
use speciality <- decode.field("speciality", pocket_monster_type_decoder)
decode.success(GymLeader(name, speciality))
}
A third decoder can be used to extract and decode the "type" field, and the
then function then returns whichever decoder is suitable for the document.
let decoder = {
use tag <- decode.field("type", decode.string)
case tag {
"gym-leader" -> gym_leader_decoder
_ -> trainer_decoder
}
}
decode.run(data, decoder)
Types
Error returned when unexpected data is encountered
pub type DecodeError {
DecodeError(
expected: String,
found: String,
path: List(String),
)
}Constructors
-
DecodeError(expected: String, found: String, path: List(String))
A decoder is a value that can be used to turn dynamically typed Dynamic
data into typed data using the run function.
Several smaller decoders can be combined to make larger decoders using
functions such as list and field.
pub opaque type Decoder(t)Constants
pub const bit_array: Decoder(BitArray)A decoder that decodes BitArray values. This decoder never returns an error.
Examples
let result = decode.run(dynamic.from(<<5, 7>>), decode.bit_array)
assert result == Ok(<<5, 7>>)
pub const bool: Decoder(Bool)A decoder that decodes Bool values.
Examples
let result = decode.run(dynamic.from(True), decode.bool)
assert result == Ok(True)
pub const dynamic: Decoder(Dynamic)A decoder that decodes Dynamic values. This decoder never returns an error.
Examples
let result = decode.run(dynamic.from(3.14), decode.dynamic)
assert result == Ok(dynamic.from(3.14))
pub const float: Decoder(Float)A decoder that decodes Float values.
Examples
let result = decode.run(dynamic.from(3.14), decode.float)
assert result == Ok(3.14)
pub const int: Decoder(Int)A decoder that decodes Int values.
Examples
let result = decode.run(dynamic.from(147), decode.int)
assert result == Ok(147)
Functions
pub fn at(path: List(a), inner: Decoder(b)) -> Decoder(b)A decoder that decodes a value that is nested within other values. For example, decoding a value that is within some deeply nested JSON objects.
This function will index into dictionaries with any key type, and if the key is an int then it’ll also index into Erlang tuples and JavaScript arrays, and the first two elements of Gleam lists.
Examples
let decoder = decode.at(["one", "two"], decode.int)
let data = dynamic.from(dict.from_list([
#("one", dict.from_list([
#("two", 1000),
])),
]))
decode.run(data, decoder)
// -> Ok(1000)
dynamic.from(Nil)
|> decode.run(decode.optional(decode.int))
// -> Ok(option.None)
pub fn collapse_errors(
decoder: Decoder(a),
name: String,
) -> Decoder(a)Replace all errors produced by a decoder with one single error for a named expected type.
This function may be useful if you wish to simplify errors before
presenting them to a user, particularly when using the one_of function.
Examples
let decoder = decode.string |> decode.collapse_errors("MyThing")
let result = decode.run(dynamic.from(1000), decoder)
assert result == Error([DecodeError("MyThing", "Int", [])])
pub fn decode_error(
expected expected: String,
found found: Dynamic,
) -> List(DecodeError)Construct a decode error for some unexpected dynamic data.
pub fn dict(
key: Decoder(a),
value: Decoder(b),
) -> Decoder(Dict(a, b))A decoder that decodes dicts where all keys and vales are decoded with given decoders.
Examples
let values = dict.from_list([
#("one", 1),
#("two", 2),
])
let result =
decode.run(dynamic.from(values), decode.dict(decode.string, decode.int))
assert result == Ok(values)
pub fn failure(zero: a, expected: String) -> Decoder(a)Define a decoder that always fails. The parameter for this function is the name of the type that has failed to decode.
pub fn field(
field_name: a,
field_decoder: Decoder(b),
next: fn(b) -> Decoder(c),
) -> Decoder(c)Run a decoder on a field of a Dynamic value, decoding the value if it is
of the desired type, or returning errors. An error is returned if there is
no field for the specified key.
This function will index into dictionaries with any key type, and if the key is an int then it’ll also index into Erlang tuples and JavaScript arrays, and the first two elements of Gleam lists.
Examples
let data = dynamic.from(dict.from_list([
#("email", "lucy@example.com"),
#("name", "Lucy"),
]))
let decoder = {
use name <- decode.field("name", string)
use email <- decode.field("email", string)
SignUp(name: name, email: email)
}
let result = decode.run(data, decoder)
assert result == Ok(SignUp(name: "Lucy", email: "lucy@example.com"))
If you wish to decode a value that is more deeply nested within the dynamic
data, see subfield and at.
If you wish to return a default in the event that a field is not present,
see optional_field and / optionally_at.
pub fn list(of inner: Decoder(a)) -> Decoder(List(a))A decoder that decodes lists where all elements are decoded with a given decoder.
Examples
let result =
decode.run(dynamic.from([1, 2, 3]), decode.list(of: decode.int))
assert result == Ok([1, 2, 3])
pub fn map(
decoder: Decoder(a),
transformer: fn(a) -> b,
) -> Decoder(b)Apply a transformation function to any value decoded by the decoder.
Examples
let decoder = decode.int |> decode.map(int.to_string)
let result = decode.run(dynamic.from(1000), decoder)
assert result == Ok("1000")
pub fn map_errors(
decoder: Decoder(a),
transformer: fn(List(DecodeError)) -> List(DecodeError),
) -> Decoder(a)Apply a transformation function to any errors returned by the decoder.
pub fn new_primitive_decoder(
name: String,
decoding_function: fn(Dynamic) -> Result(a, a),
) -> Decoder(a)Create a decoder for a new data type from a decoding function.
This function is used for new primitive types. For example, you might define a decoder for Erlang’s pid type.
A default “zero” value is also required to make a decoder. When this decoder is used as part of a larger decoder this zero value used as a placeholder so that the rest of the decoder can continue to run and collect all decoding errors.
If you were to make a decoder for the String type (rather than using the
build-in string decoder) you would define it like so:
import gleam/dynamic
import decode/decode
pub fn string_decoder() -> decode.Decoder(String) {
let default = ""
decode.new_primitive_decoder("String", fn(data) {
case dynamic.string {
Ok(x) -> Ok(x)
Error(x) -> Error(default)
}
})
}
pub fn one_of(
first: Decoder(a),
or alternatives: List(Decoder(a)),
) -> Decoder(a)Create a new decoder from several other decoders. Each of the inner decoders is run in turn, and the value from the first to succeed is used.
If no decoder succeeds then the errors from the first decoder is used.
If you wish for different errors then you may wish to use the
collapse_errors or map_errors functions.
Examples
let decoder = decode.one_of(decode.string, or: [
decode.int |> decode.map(int.to_string),
decode.float |> decode.map(float.to_string),
])
decode.run(dynamic.from(1000), decoder)
// -> Ok("1000")
pub fn optional(inner: Decoder(a)) -> Decoder(Option(a))A decoder that decodes nullable values of a type decoded by with a given decoder.
This function can handle common representations of null on all runtimes, such as
nil, null, and undefined on Erlang, and undefined and null on
JavaScript.
Examples
let result = decode.run(dynamic.from(100), decode.optional(decode.int))
assert result == Ok(option.Some(100))
let result = decode.run(dynamic.from(Nil), decode.optional(decode.int))
assert result == Ok(option.None)
pub fn optional_field(
key: a,
default: b,
field_decoder: Decoder(b),
next: fn(b) -> Decoder(c),
) -> Decoder(c)Run a decoder on a field of a Dynamic value, decoding the value if it is
of the desired type, or returning errors. The given default value is
returned if there is no field for the specified key.
This function will index into dictionaries with any key type, and if the key is an int then it’ll also index into Erlang tuples and JavaScript arrays, and the first two elements of Gleam lists.
Examples
let data = dynamic.from(dict.from_list([
#("name", "Lucy"),
]))
let decoder = {
use name <- decode.field("name", string)
use email <- decode.optional_field("email", "n/a", string)
SignUp(name: name, email: email)
}
let result = decode.run(data, decoder)
assert result == Ok(SignUp(name: "Lucy", email: "n/a"))
pub fn optionally_at(
path: List(a),
default: b,
inner: Decoder(b),
) -> Decoder(b)A decoder that decodes a value that is nested within other values. For example, decoding a value that is within some deeply nested JSON objects.
This function will index into dictionaries with any key type, and if the key is an int then it’ll also index into Erlang tuples and JavaScript arrays, and the first two elements of Gleam lists.
Examples
let decoder = decode.optionally_at(["one", "two"], 100, decode.int)
let data = dynamic.from(dict.from_list([
#("one", dict.from_list([])),
]))
decode.run(data, decoder)
// -> Ok(100)
pub fn recursive(inner: fn() -> Decoder(a)) -> Decoder(a)Create a decoder that can refer to itself, useful for decoding for deeply nested data.
Attempting to create a recursive decoder without this function could result
in an infinite loop. If you are using field or other useable function
then you may not need to use this function.
import gleam/dynamic
import decode/zero.{type Decoder}
type Nested {
Nested(List(Nested))
Value(String)
}
fn nested_decoder() -> Decoder(Nested) {
use <- zero.recursive
zero.one_of(zero.string |> zero.map(Value), [
zero.list(nested_decoder()) |> zero.map(Nested),
])
}
pub fn run(
data: Dynamic,
decoder: Decoder(a),
) -> Result(a, List(DecodeError))Run a decoder on a Dynamic value, decoding the value if it is of the
desired type, or returning errors.
Examples
let decoder = {
use name <- decode.field("email", decode.string)
use email <- decode.field("password", decode.string)
decode.success(SignUp(name: name, email: email))
}
decode.run(data, decoder)
pub fn subfield(
field_path: List(a),
field_decoder: Decoder(b),
next: fn(b) -> Decoder(c),
) -> Decoder(c)The same as field, except taking a path to the value rather
than a field name.
This function will index into dictionaries with any key type, and if the key is an int then it’ll also index into Erlang tuples and JavaScript arrays, and the first two elements of Gleam lists.
Examples
let data = dynamic.from(dict.from_list([
#("data", dict.from_list([
#("email", "lucy@example.com"),
#("name", "Lucy"),
]))
]))
let decoder = {
use name <- decode.subfield(["data", "name"], decode.string)
use email <- decode.subfield(["data", "email"], decode.string)
decode.success(SignUp(name: name, email: email))
}
let result = decode.run(data, decoder)
assert result == Ok(SignUp(name: "Lucy", email: "lucy@example.com"))
pub fn success(data: a) -> Decoder(a)Finalise a decoder having successfully extracted a value.
Examples
let data = dynamic.from(dict.from_list([
#("email", "lucy@example.com"),
#("name", "Lucy"),
]))
let decoder = {
use name <- decode.field("name", string)
use email <- decode.field("email", string)
decode.success(SignUp(name: name, email: email))
}
let result = decode.run(data, decoder)
assert result == Ok(SignUp(name: "Lucy", email: "lucy@example.com"))