View Source NeoSaxy (NeoSaxy v1.0.2)

NeoSaxy is an XML SAX parser and encoder.

NeoSaxy provides functions to parse XML file in both binary and streaming way in compliant with Extensible Markup Language (XML) 1.0 (Fifth Edition).

NeoSaxy also offers DSL and API to build, compose and encode XML document. See "Encoder" section below for more information.

Parser

NeoSaxy parser supports two modes of parsing: SAX and simple form.

SAX mode (Simple API for XML)

SAX is an event driven algorithm for parsing XML documents. A SAX parser takes XML document as the input and emits events out to a pre-configured event handler during parsing.

There are several types of SAX events supported by NeoSaxy:

:start_document - after prolog is parsed.
:start_element - when open tag is parsed.
:characters - when a chunk of CharData is parsed.
:cdata - when a chunk of CData is parsed.
:end_element - when end tag is parsed.
:end_document - when the root element is closed.

See NeoSaxy.Handler for more information.

Simple form mode

NeoSaxy supports parsing XML documents into a simple format. See NeoSaxy.SimpleForm for more details.

Encoding

NeoSaxy only supports UTF-8 encoding. It also respects the encoding set in XML document prolog, which means that if the declared encoding is not UTF-8, the parser stops. Anyway, when there is no encoding declared, NeoSaxy defaults the encoding to UTF-8.

Reference expansion

NeoSaxy supports expanding character references and XML 1.0 predefined entity references, for example A is expanded to "A", & to "&", and & to "&".

NeoSaxy does not expand external entity references, but provides an option to specify how they should be handled. See more in "Shared options" section.

Creation of atoms

NeoSaxy does not create atoms during the parsing process.

DTD and XSD

NeoSaxy does not support parsing DTD (Doctype Definition) and XSD schemas. When encountering DTD, the parser simply skips that.

Shared options

:expand_entity - specifies how external entity references should be handled. Three supported strategies respectively are:
- :keep - keep the original binary, for example Orange ® will be expanded to "Orange ®", this is the default strategy.
- :skip - skip the original binary, for example Orange ® will be expanded to "Orange ".
- {mod, fun, args} - take the applied result of the specified MFA.
- :never - keep the original binary, including predefined entity reference, e.g. "Orange &" will remain "Orange &"
:cdata_as_characters - true to emit CData events as :characters. Defaults to true.

Encoder

NeoSaxy offers two APIs to build simple form and encode XML document.

Use NeoSaxy.XML to build and compose XML simple form, then NeoSaxy.encode!/2 to encode the built element into XML binary.

iex> import NeoSaxy.XML
iex> element = element("person", [gender: "female"], "Alice")
{"person", [{"gender", "female"}], ["Alice"]}
iex> NeoSaxy.encode!(element, [version: "1.0"])
"<?xml version=\"1.0\"?><person gender=\"female\">Alice</person>"

See NeoSaxy.XML for more XML building APIs.

NeoSaxy also provides NeoSaxy.Builder protocol to help composing structs into simple form.

defmodule Person do
  @derive {
    NeoSaxy.Builder,
    name: "person", attributes: [:gender], children: [:name, emails: &__MODULE__.build_emails/1]
  }

  import NeoSaxy.XML

  defstruct [:name, :gender, emails: []]

  def build_emails(emails) do
    count = Enum.count(emails)

    element(
      "emails",
      [count: Enum.count(emails)],
      Enum.map(emails, &element("email", [], &1))
    )
  end
end

iex> jack = %Person{gender: :male, name: "Jack", emails: ["john@example.com"]}
iex> john = %Person{gender: :male, name: "John"}
iex> import NeoSaxy.XML
iex> root = element("people", [], [jack, john])
iex> NeoSaxy.encode!(root, [version: "1.0"])
"<?xml version=\"1.0\"?><people><person gender=\"male\">Jack<emails count=\"1\"><email>john@example.com</email></emails></person><person gender=\"male\">John<emails count=\"0\"/></person></people>"

Summary

Functions

encode!(root, prolog \\ nil)

Encodes a simple form XML element into string.

encode_to_iodata!(root, prolog \\ nil)

Encodes a simple form element into IO data.

parse_stream(stream, handler, initial_state, options \\ [])

Parses XML stream data.

parse_string(data, handler, initial_state, options \\ [])

Parses XML binary data.

stream_events(stream, options \\ [])

Parses XML stream and returns a stream of elements.

Functions

encode!(root, prolog \\ nil)

@spec encode!(
  root :: NeoSaxy.XML.element(),
  prolog :: NeoSaxy.Prolog.t() | Keyword.t() | nil
) ::
  String.t()

Encodes a simple form XML element into string.

This function encodes an element in simple form format and a prolog to an XML document.

Examples

iex> import NeoSaxy.XML
iex> root = element(:foo, [{"foo", "bar"}], "bar")
iex> prolog = [version: "1.0"]
iex> NeoSaxy.encode!(root, prolog)
"<?xml version=\"1.0\"?><foo foo=\"bar\">bar</foo>"
iex> prolog = [version: "1.0", encoding: "UTF-8"]
iex> NeoSaxy.encode!(root, prolog)
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><foo foo=\"bar\">bar</foo>"

encode_to_iodata!(root, prolog \\ nil)

@spec encode_to_iodata!(
  root :: NeoSaxy.XML.element(),
  prolog :: NeoSaxy.Prolog.t() | Keyword.t() | nil
) ::
  iodata()

Encodes a simple form element into IO data.

Same as encode!/2 but this encodes the document into IO data.

Examples

iex> import NeoSaxy.XML
iex> root = element(:foo, [{"foo", "bar"}], "bar")
iex> prolog = [version: "1.0"]
iex> NeoSaxy.encode_to_iodata!(root, prolog)
[
  [~c'<?xml', [32, ~c'version', 61, 34, "1.0", 34], [], [], ~c'?>'],
  [60, "foo", 32, "foo", 61, 34, "bar", 34],
  62,
  ["bar"],
  [60, 47, "foo", 62]
]

parse_stream(stream, handler, initial_state, options \\ [])

@spec parse_stream(
  stream :: Enumerable.t(),
  handler :: module(),
  initial_state :: term(),
  options :: Keyword.t()
) ::
  {:ok, state :: term()}
  | {:halt, state :: term(), rest :: String.t()}
  | {:error, exception :: NeoSaxy.ParseError.t()}

Parses XML stream data.

This function takes a stream, SAX event handler (see more at NeoSaxy.Handler) and an initial state as the input, it returns {:ok, state} if parsing is successful, otherwise {:error, exception}, where exception is a NeoSaxy.ParseError struct which can be converted into readable message with Exception.message/1.

Examples

defmodule MyTestHandler do
  @behaviour NeoSaxy.Handler

  def handle_event(:start_document, prolog, state) do
    {:ok, [{:start_document, prolog} | state]}
  end

  def handle_event(:end_document, _data, state) do
    {:ok, [{:end_document} | state]}
  end

  def handle_event(:start_element, {name, attributes}, state) do
    {:ok, [{:start_element, name, attributes} | state]}
  end

  def handle_event(:end_element, name, state) do
    {:ok, [{:end_element, name} | state]}
  end

  def handle_event(:characters, chars, state) do
    {:ok, [{:chacters, chars} | state]}
  end
end

iex> stream = File.stream!("./test/support/fixture/foo.xml")
iex> NeoSaxy.parse_stream(stream, MyTestHandler, [])
{:ok,
 [{:end_document},
  {:end_element, "foo"},
  {:start_element, "foo", [{"bar", "value"}]},
  {:start_document, [version: "1.0"]}]}

Memory usage

NeoSaxy.parse_stream/3 takes a File.Stream or Stream as the input, so the amount of bytes to buffer in each chunk can be controlled by File.stream!/3 API.

During parsing, the actual memory used by NeoSaxy might be higher than the number configured for each chunk, since NeoSaxy holds in memory some parsed parts of the original binary to leverage Erlang sub-binary extracting. Anyway, NeoSaxy tries to free those up when it makes sense.

Options

See the “Shared options” section at the module documentation.

:character_data_max_length - tells the parser to emit the :characters event when its length exceeds the specified number. The option is useful when the tag being parsed containing a very large chunk of data. Defaults to :infinity.

parse_string(data, handler, initial_state, options \\ [])

@spec parse_string(
  data :: binary(),
  handler :: module(),
  initial_state :: term(),
  options :: Keyword.t()
) ::
  {:ok, state :: term()}
  | {:halt, state :: term(), rest :: String.t()}
  | {:error, exception :: NeoSaxy.ParseError.t()}

Parses XML binary data.

This function takes XML binary, SAX event handler (see more at NeoSaxy.Handler) and an initial state as the input, it returns {:ok, state} if parsing is successful, otherwise {:error, exception}, where exception is a NeoSaxy.ParseError struct which can be converted into readable message with Exception.message/1.

The third argument state can be used to keep track of data and parsing progress when parsing is happening, which will be returned when parsing finishes.

Options

See the “Shared options” section at the module documentation.

Examples

defmodule MyTestHandler do
  @behaviour NeoSaxy.Handler

  def handle_event(:start_document, prolog, state) do
    {:ok, [{:start_document, prolog} | state]}
  end

  def handle_event(:end_document, _data, state) do
    {:ok, [{:end_document} | state]}
  end

  def handle_event(:start_element, {name, attributes}, state) do
    {:ok, [{:start_element, name, attributes} | state]}
  end

  def handle_event(:end_element, name, state) do
    {:ok, [{:end_element, name} | state]}
  end

  def handle_event(:characters, chars, state) do
    {:ok, [{:chacters, chars} | state]}
  end
end

iex> xml = "<?xml version='1.0' ?><foo bar='value'></foo>"
iex> NeoSaxy.parse_string(xml, MyTestHandler, [])
{:ok,
 [{:end_document},
  {:end_element, "foo"},
  {:start_element, "foo", [{"bar", "value"}]},
  {:start_document, [version: "1.0"]}]}

stream_events(stream, options \\ [])

@spec stream_events(in_stream :: Enumerable.t(), options :: Keyword.t()) ::
  out_stream :: Enumerable.t()

Parses XML stream and returns a stream of elements.

This function takes a stream and returns a stream of xml SAX events. When any parsing error occurs, it raises a NeoSaxy.ParseError exception.

Examples

iex> stream = File.stream!("./test/support/fixture/foo.xml")
iex> Enum.to_list NeoSaxy.stream_events stream
[
  start_document: [version: "1.0"],
  start_element: {"foo", [{"bar", "value"}]},
  end_element: "foo"
]
iex> Enum.to_list NeoSaxy.stream_events ["<foo>unclosed value"]
** (NeoSaxy.ParseError) unexpected end of input, expected token: :chardata

Warning
Input stream is evaluated lazily, therefore some events may be emitted before exception is raised

Memory usage

NeoSaxy.stream_events/2 takes a File.Stream or Stream as the input, so the amount of bytes to buffer in each chunk can be controlled by File.stream!/3 API.

Options

See the “Shared options” section at the module documentation.

:character_data_max_length - tells the parser to emit the :characters event when its length exceeds the specified number. The option is useful when the tag being parsed containing a very large chunk of data. Defaults to :infinity.