RustyJson (rustyjson v0.3.9)

Copy Markdown View Source

A high-performance JSON library for Elixir powered by Rust NIFs.

RustyJson is designed as a drop-in replacement for Jason with significantly better performance characteristics:

  • 3-6x faster encoding for medium/large payloads
  • 10-20x lower memory usage during encoding
  • Full JSON spec compliance (RFC 8259)
  • Native type support for DateTime, Decimal, URI, and more

Quick Start

# Encoding
iex> RustyJson.encode!(%{name: "Alice", age: 30})
~s({"age":30,"name":"Alice"})

# Decoding
iex> RustyJson.decode!(~s({"name":"Alice","age":30}))
%{"age" => 30, "name" => "Alice"}

# Pretty printing
iex> RustyJson.encode!(%{items: [1, 2, 3]}, pretty: true)
"""
{
  "items": [
    1,
    2,
    3
  ]
}
"""

Phoenix Integration

Configure Phoenix to use RustyJson as the JSON library:

# config/config.exs
config :phoenix, :json_library, RustyJson

Why RustyJson?

Traditional JSON libraries in Elixir create many intermediate binary allocations during encoding, which pressures the garbage collector. RustyJson eliminates this by walking the Erlang term tree directly in Rust and writing to a single buffer.

For a detailed comparison, see the README.

Module Overview

ModuleDescription
RustyJsonMain encoding/decoding API
RustyJson.EncoderProtocol for custom type encoding
RustyJson.EncodeLow-level encoding functions
RustyJson.FragmentPre-encoded JSON injection
RustyJson.FormatterJSON pretty-printing utilities
RustyJson.HelpersCompile-time JSON macros (json_map, json_map_take)
RustyJson.Sigil~j/~J sigils for JSON literals
RustyJson.OrderedObjectOrder-preserving JSON object (for objects: :ordered_objects)
RustyJson.DecoderJSON decoding module (Jason.Decoder compatible)
RustyJson.DecodeErrorDecoding error exception
RustyJson.EncodeErrorEncoding error exception

Built-in Type Support

RustyJson natively handles these Elixir types without protocol overhead:

Elixir TypeJSON OutputExample
mapobject%{a: 1}{"a":1}
listarray[1, 2][1,2]
tuplearray{1, 2}[1,2]
binarystring"hello""hello"
integernumber4242
floatnumber3.143.14
true/falsebooleantruetrue
nilnullnilnull
atomstring:hello"hello"
DateTimeISO8601 string~U[2024-01-15 14:30:00Z]"2024-01-15T14:30:00Z"
NaiveDateTimeISO8601 string~N[2024-01-15 14:30:00]"2024-01-15T14:30:00"
DateISO8601 string~D[2024-01-15]"2024-01-15"
TimeISO8601 string~T[14:30:00]"14:30:00"
DecimalstringDecimal.new("123.45")"123.45"
URIstringURI.parse("https://example.com")"https://example.com"
structsobject%User{name: "Alice"}{"name":"Alice"} (requires @derive RustyJson.Encoder or explicit defimpl)

Note: MapSet and Range are not natively encoded. They require an explicit RustyJson.Encoder implementation or protocol: false to encode via the Rust NIF directly. This matches Jason's behavior.

Escape Modes

RustyJson supports multiple escape modes for different security contexts:

ModeDescriptionUse Case
:jsonStandard JSON escaping (default)General use
:html_safeEscapes <, >, & as \uXXXX and / as \/HTML embedding
:javascript_safeEscapes line/paragraph separatorsJavaScript strings
:unicode_safeEscapes all non-ASCII as \uXXXXASCII-only output

Performance Tips

  1. Bypass the protocol: The protocol is enabled by default for Jason compatibility. If you have no custom RustyJson.Encoder implementations, use protocol: false to bypass protocol dispatch for maximum speed.

  2. Use lean mode: If you don't have DateTime/Decimal types, use lean: true to skip struct type detection in Rust.

  3. Use compression: For large payloads over the network, compress: :gzip reduces output size 5-10x.

  4. Avoid keys: :atoms with untrusted input: keys: :atoms uses String.to_atom/1, which can exhaust the atom table. Use keys: :atoms! (which uses String.to_existing_atom/1) or keys: :strings (default) instead.

  5. Use key interning for bulk data: When decoding arrays of objects with the same schema (API responses, database results, webhooks), use keys: :intern for ~30% faster parsing:

    RustyJson.decode!(json, keys: :intern)

    The intern cache is capped at 4096 unique keys. Beyond that, new keys are allocated normally — this bounds worst-case CPU time and stops paying cache overhead when keys aren't being reused.

    Caution: Don't use for single objects or varied schemas—cache overhead makes it 2-3x slower when keys aren't reused.

Error Handling

RustyJson provides clear, actionable error messages. encode/1 and decode/1 consistently return {:error, reason} tuples for invalid input.

# Error messages describe the problem
RustyJson.decode(~s({"key": "value\\'s"}))
# => {:error, "Invalid escape sequence: \\'"}

# Unencodable values return error tuples
RustyJson.encode(%{{:tuple, :key} => 1})
# => {:error, "Map key must be atom, string, or integer"}

# Strict UTF-16 surrogate validation per RFC 7493
RustyJson.decode(~s("\\uD800"))
# => {:error, "Lone surrogate in string"}

This makes error handling predictable—pattern match on results without needing try/rescue blocks.

Summary

Types

Supported compression algorithms for encoding.

Compression level from 0 (fastest, least compression) to 9 (slowest, best compression).

Compression options tuple.

Internal compression options format passed to NIF.

Options for decode/2 and decode!/2.

Options for encode/2 and encode!/2.

Escape mode for JSON string encoding.

Options for decoding JSON object keys.

Functions

Decodes a JSON string to an Elixir term.

Decodes a JSON string to an Elixir term, raising on error.

Encodes an Elixir term to a JSON string.

Encodes an Elixir term to a JSON string, raising on error.

Encodes a term to iodata (for Phoenix compatibility).

Encodes a term to iodata, raising on error (for Phoenix compatibility).

Types

compression_algorithm()

@type compression_algorithm() :: :gzip | :none

Supported compression algorithms for encoding.

  • :gzip - Standard gzip compression
  • :none - No compression (default)

compression_level()

@type compression_level() :: 0..9

Compression level from 0 (fastest, least compression) to 9 (slowest, best compression).

compression_option()

@type compression_option() :: :gzip | {:gzip, compression_level()} | :none

Compression options tuple.

Can be specified as:

  • :gzip - Use default compression level
  • {:gzip, level} - Use specific compression level (0-9)
  • :none - No compression

compression_options()

@type compression_options() :: {compression_algorithm(), compression_level() | nil}

Internal compression options format passed to NIF.

decode_opt()

@type decode_opt() ::
  {:keys, keys()}
  | {:strings, :copy | :reference}
  | {:objects, :maps | :ordered_objects}
  | {:floats, :native | :decimals}
  | {:decoding_integer_digit_limit, non_neg_integer()}
  | {:max_bytes, non_neg_integer()}
  | {:duplicate_keys, :last | :error}
  | {:validate_strings, boolean()}
  | {:dirty_threshold, non_neg_integer()}

Options for decode/2 and decode!/2.

  • :keys - How to handle object keys (see keys/0). Default: :strings
  • :strings - How to handle decoded strings. :copy or :reference. Both produce copies (RustyJson always copies). Default: :reference
  • :objects - How to decode JSON objects. :maps (default) or :ordered_objects
  • :floats - How to decode JSON floats. :native (default) or :decimals
  • :decoding_integer_digit_limit - Maximum digits in integer part. 0 disables. Default: 1024, or the value of Application.compile_env(:rustyjson, :decoding_integer_digit_limit)
  • :max_bytes - Maximum input size in bytes. 0 means unlimited (default). The check is performed using IO.iodata_length/1 before converting to binary, avoiding the memory spike from allocating the full binary.
  • :duplicate_keys - How to handle duplicate object keys. :last (default) uses last-wins semantics. :error rejects objects with duplicate keys. Performance note: :error adds per-key overhead from HashSet tracking. Use only when strict validation is needed.
  • :validate_strings - Whether to validate that decoded strings contain valid UTF-8. Default: true. When true, rejects strings with invalid UTF-8 byte sequences. Set to false to skip validation for maximum throughput on trusted input.
  • :dirty_threshold - Byte size threshold for auto-dispatching to dirty CPU scheduler. When input size >= this threshold, decode runs on a dirty scheduler to avoid blocking normal BEAM schedulers. Default: 102400 (100KB). Set to 0 to disable.

encode_opt()

@type encode_opt() ::
  {:pretty, boolean() | pos_integer() | keyword()}
  | {:escape, escape_mode()}
  | {:compress, compression_option()}
  | {:protocol, boolean()}
  | {:lean, boolean()}
  | {:maps, :naive | :strict}
  | {:sort_keys, boolean()}
  | {:scheduler, :auto | :normal | :dirty}

Options for encode/2 and encode!/2.

  • :pretty - Pretty print with indentation. true for 2 spaces, an integer for custom spacing, a string/iodata for custom indent (e.g. "\t" for tabs), or a keyword list with :indent, :line_separator, and :after_colon keys.
  • :escape - Escape mode (see escape_mode/0). Default: :json
  • :compress - Compression (see compression_option/0). Default: :none
  • :protocol - Use RustyJson.Encoder protocol. Default: true
  • :lean - Skip special struct handling. Default: false
  • :maps - Key uniqueness mode. :naive (default) allows duplicate serialized keys, :strict raises on duplicate keys (e.g. atom :a and string "a" in the same map).

escape_mode()

@type escape_mode() :: :json | :html_safe | :javascript_safe | :unicode_safe

Escape mode for JSON string encoding.

  • :json - Standard JSON escaping (default)
  • :html_safe - Also escape <, >, &, / for safe HTML embedding
  • :javascript_safe - Also escape line/paragraph separators (U+2028, U+2029)
  • :unicode_safe - Escape all non-ASCII characters as \uXXXX

keys()

@type keys() :: :strings | :atoms | :atoms! | :copy | :intern | (String.t() -> term())

Options for decoding JSON object keys.

  • :strings - Keep keys as strings (default, safe)
  • :atoms - Convert to atoms using String.to_atom/1 (unsafe with untrusted input)
  • :atoms! - Convert to existing atoms using String.to_existing_atom/1 (safe, raises if missing)
  • :copy - Copy key binaries (same as :strings in RustyJson since NIFs always copy)
  • :intern - Cache repeated keys during parsing (~30% faster for arrays of objects). The cache is capped at 4096 unique keys — beyond this, new keys are allocated normally. This bounds worst-case CPU time and avoids cache overhead for pathological inputs with many distinct keys. The cap is internal and not user-configurable.
  • A function of arity 1 - Applied to each key string recursively

Functions

decode(input, opts \\ [])

@spec decode(iodata(), [decode_opt()]) ::
  {:ok, term()} | {:error, RustyJson.DecodeError.t()}

Decodes a JSON string to an Elixir term.

Returns {:ok, term} on success or {:error, reason} on failure.

Options

  • :keys - How to decode object keys. One of:

    • :strings - Keep as strings (default, safe)
    • :atoms - Convert to atoms (unsafe with untrusted input)
    • :atoms! - Convert to existing atoms only (safe, raises if atom missing)
    • :copy - Copy key binaries (equivalent to :strings in RustyJson)
    • :intern - Cache repeated keys during parsing. ~30% faster for arrays of objects with the same schema (REST APIs, GraphQL, database results, webhooks). The cache is capped at 4096 unique keys; beyond that, new keys are allocated normally (no worse than default). This bounds worst-case CPU time and stops paying cache overhead when keys aren't being reused. Caution: 2-3x slower for single objects or varied schemas—only use for homogeneous arrays of 10+ objects.
    • A function of arity 1 - Applied recursively to each key string. Example: keys: &String.upcase/1
  • :strings - How to handle decoded strings. :reference (default) or :copy. Both produce copies in RustyJson (Rust NIFs always copy into BEAM binaries), so this option exists for Jason API compatibility.

  • :objects - How to decode JSON objects. :maps (default) or :ordered_objects. When :ordered_objects, returns %RustyJson.OrderedObject{} structs that preserve key insertion order.

  • :floats - How to decode JSON floats. :native (default) returns Elixir floats, :decimals returns %Decimal{} structs for exact decimal representation.

  • :decoding_integer_digit_limit - Maximum number of digits allowed in the integer part of a JSON number. Integers exceeding this limit cause a decode error. Default: 1024, or the value of Application.compile_env(:rustyjson, :decoding_integer_digit_limit). Set to 0 to disable the limit.

Examples

iex> RustyJson.decode(~s({"name":"Alice","age":30}))
{:ok, %{"age" => 30, "name" => "Alice"}}

iex> RustyJson.decode(~s([1, 2, 3]))
{:ok, [1, 2, 3]}

iex> RustyJson.decode("invalid")
{:error, "Unexpected character at position 0"}

Security Considerations

Avoid keys: :atoms with untrusted input. Atoms are not garbage collected, so an attacker could exhaust your atom table by sending JSON with many unique keys.

Use keys: :atoms! if you expect specific keys to exist, or keys: :strings (default).

See decode!/2 for a version that raises on error.

decode!(input, opts \\ [])

@spec decode!(iodata(), [decode_opt()]) :: term()

Decodes a JSON string to an Elixir term, raising on error.

Same as decode/2 but raises RustyJson.DecodeError on failure. The raised exception includes :data, :position, and :token fields for detailed error diagnostics.

Options

See decode/2 for available options.

Examples

iex> RustyJson.decode!(~s({"x": [1, 2, 3]}))
%{"x" => [1, 2, 3]}

iex> RustyJson.decode!(~s({"x": 1}), keys: :atoms)
%{x: 1}

iex> RustyJson.decode!(~s({"x": 1}), keys: &String.upcase/1)
%{"X" => 1}

iex> RustyJson.decode!("null")
nil

iex> RustyJson.decode!("true")
true

iex> RustyJson.decode!(~s({"price":19.99}), floats: :decimals)
%{"price" => Decimal.new("19.99")}

JSON Types to Elixir

JSONElixir
objectmap (or RustyJson.OrderedObject with objects: :ordered_objects)
arraylist
stringbinary
number (int)integer
number (float)float (or Decimal with floats: :decimals)
truetrue
falsefalse
nullnil

Error Cases

iex> RustyJson.decode!("invalid")
** (RustyJson.DecodeError) Unexpected character at position 0

iex> RustyJson.decode!("{trailing: comma,}")
** (RustyJson.DecodeError) Expected string key at position 1

encode(input, opts \\ [])

@spec encode(term(), [encode_opt()]) ::
  {:ok, String.t()} | {:error, RustyJson.EncodeError.t() | Exception.t()}

Encodes an Elixir term to a JSON string.

Returns {:ok, json} on success or {:error, reason} on failure.

Options

  • :pretty - Pretty print with indentation. true uses 2 spaces, or pass an integer for custom spacing. Default: false

  • :escape - Escape mode for special characters. One of :json (default), :html_safe, :javascript_safe, or :unicode_safe. See escape_mode/0.

  • :compress - Compression algorithm. :gzip or {:gzip, 0..9} for specific level. Default: :none

  • :protocol - Enable RustyJson.Encoder protocol for custom types. Default: true

  • :lean - Skip struct type detection (DateTime, Decimal, etc. encoded as raw maps). Default: false

  • :maps - Key uniqueness mode. :naive (default) allows duplicate serialized keys, :strict errors on duplicate keys (e.g. atom :a and string "a").

  • :sort_keys - Sort map keys lexicographically in output. Default: false. JSON objects are unordered per RFC 8259. Enable for deterministic output (useful for snapshot tests, caching, or diffing). Note: Jason always sorts keys; RustyJson does not by default for performance.

Examples

iex> RustyJson.encode(%{name: "Alice", scores: [95, 87, 92]})
{:ok, ~s({"name":"Alice","scores":[95,87,92]})}

iex> RustyJson.encode(%{valid: true}, pretty: true)
{:ok, "{\n  \"valid\": true\n}"}

iex> RustyJson.encode("invalid UTF-8: " <> <<0xFF>>)
{:error, "Failed to decode binary as UTF-8"}

Error Handling

Common error cases:

  • Invalid UTF-8 binary
  • Non-finite float (NaN, Infinity)
  • Circular references (will cause stack overflow)

See encode!/2 for a version that raises on error.

encode!(input, opts \\ [])

@spec encode!(term(), [encode_opt()]) :: String.t()

Encodes an Elixir term to a JSON string, raising on error.

Same as encode/2 but raises RustyJson.EncodeError on failure.

Options

See encode/2 for available options.

Examples

iex> RustyJson.encode!(%{hello: "world"})
~s({"hello":"world"})

iex> RustyJson.encode!([1, 2, 3], pretty: 4)
"""
[
    1,
    2,
    3
]
"""

iex> RustyJson.encode!(~D[2024-01-15])
~s("2024-01-15")

iex> RustyJson.encode!(%{html: "<script>"}, escape: :html_safe)
~s({"html":"\u003cscript\u003e"})

Custom Encoding

For custom types, implement RustyJson.Encoder. The protocol is used by default (protocol: true). Implementations should return iodata via RustyJson.Encode functions for best performance:

defmodule Money do
  defstruct [:amount, :currency]
end

defimpl RustyJson.Encoder, for: Money do
  def encode(%Money{amount: amount, currency: currency}, opts) do
    RustyJson.Encode.map(%{amount: Decimal.to_string(amount), currency: currency}, opts)
  end
end

iex> money = %Money{amount: Decimal.new("99.99"), currency: "USD"}
iex> RustyJson.encode!(money)
~s({"amount":"99.99","currency":"USD"})

For backwards compatibility, returning a plain map is also supported and will be re-encoded automatically (with a small overhead).

Compression

iex> json = RustyJson.encode!(%{data: String.duplicate("x", 1000)}, compress: :gzip)
iex> :zlib.gunzip(json)
~s({"data":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"})

encode_to_iodata(input, opts \\ [])

@spec encode_to_iodata(term(), [encode_opt()]) ::
  {:ok, iodata()} | {:error, RustyJson.EncodeError.t() | Exception.t()}

Encodes a term to iodata (for Phoenix compatibility).

This function exists to implement the Phoenix JSON library interface. Returns {:ok, binary} on success or {:error, reason} on failure.

A Note on iodata

RustyJson returns a single binary, not an iolist. This is intentional and provides excellent performance for payloads up to ~100MB.

The memory efficiency comes from the encoding process (no intermediate allocations), not from chunked output. A 10MB payload uses ~15MB peak memory.

For truly massive payloads (100MB+), consider:

  • Streaming the data structure itself (encode in chunks)
  • Using compression (compress: :gzip reduces output 5-10x)
  • Pagination or chunked API design

Examples

iex> RustyJson.encode_to_iodata(%{status: "ok"})
{:ok, ~s({"status":"ok"})}

encode_to_iodata!(input, opts \\ [])

@spec encode_to_iodata!(term(), [encode_opt()]) :: iodata()

Encodes a term to iodata, raising on error (for Phoenix compatibility).

This function exists to implement the Phoenix JSON library interface. Raises RustyJson.EncodeError on failure.

See encode_to_iodata/2 for notes on iodata behavior.

Examples

iex> RustyJson.encode_to_iodata!(%{status: "ok"})
~s({"status":"ok"})