ToonEx.Decode.StructuralParserV2 (toon_ex v1.1.0)

Copy Markdown View Source

Structural parser for TOON format that handles indentation-based nesting.

This parser processes TOON input by analyzing indentation levels and building a hierarchical structure from the flat text representation.

Performance Design (Jason-level efficiency)

This parser is optimized for single-pass, zero-copy decoding:

  1. Binary pattern matching replaces String.starts_with?, String.contains?, String.ends_with?, and regex calls in hot paths with O(1) byte checks.
  2. Sub-binary references via binary_part/3 avoid copying — BEAM shares the underlying binary when slicing.
  3. Tail-recursive accumulators with :lists.reverse/1 instead of appending.
  4. @compile {:inline, ...} for hot functions to eliminate call overhead.
  5. Struct pattern matching in function heads replaces cond + map access.
  6. binary_part/3 for slicing instead of String.slice/3 (avoids UTF-8 scan).

Summary

Types

line_info()
parse_metadata()

Functions

parse(input, opts)

Parses TOON input string into a structured format.

Types

line_info()

@type line_info() :: %{
  content: String.t(),
  indent: non_neg_integer(),
  line_number: non_neg_integer(),
  original: String.t(),
  is_blank: boolean()
}

parse_metadata()

@type parse_metadata() :: %{
  quoted_keys: MapSet.t(String.t()),
  key_order: [String.t()]
}

Functions

parse(input, opts)

@spec parse(String.t(), map()) ::
  {:ok, {term(), parse_metadata()}} | {:error, ToonEx.DecodeError.t()}

Parses TOON input string into a structured format.

Returns a tuple of {result, metadata} where metadata contains quoted_keys and key_order.