Document is the core structure to store, manipulate, and render Markdown documents.

Tree

%MDEx.Document{
  nodes: [
    %MDEx.Paragraph{
      nodes: [
        %MDEx.Code{num_backticks: 1, literal: "Elixir"}
      ]
    }
  ]
}

Each node may contain attributes and children nodes as in the example above where MDEx.Document contains a MDEx.Paragraph node which contains a MDEx.Code node with the attributes :num_backticks and :literal.

You can check out each node's documentation in the Document Nodes section, for example MDEx.HtmlBlock.

The MDEx.Document module represents the root of a document and implements several behaviours and protocols to enable operations to fetch, update, and manipulate the document tree.

In these examples we will be using the ~MD sigil.

Tree Traversal

Understanding tree traversal is fundamental to working with MDEx documents, as it affects how all Enum functions, Access operations, and other protocols behave.

The document tree is enumerated using depth-first pre-order traversal. This means:

1. The parent node is visited first
2. Then each child node is visited recursively
3. Children are processed in the order they appear in the :nodes list

This traversal order affects all Enum functions, including Enum.at/2, Enum.map/2, Enum.find/2, and friends.

iex> doc = ~MD[# Hello]
iex> Enum.at(doc, 0)
%MDEx.Document{nodes: [%MDEx.Heading{nodes: [%MDEx.Text{literal: "Hello"}], level: 1, setext: false}]}
iex> Enum.at(doc, 1)
%MDEx.Heading{nodes: [%MDEx.Text{literal: "Hello"}], level: 1, setext: false}
iex> Enum.at(doc, 2)
%MDEx.Text{literal: "Hello"}

More complex traversal with nested elements:

iex> doc = ~MD[**bold** text]
iex> Enum.at(doc, 0)
%MDEx.Document{nodes: [%MDEx.Paragraph{nodes: [%MDEx.Strong{nodes: [%MDEx.Text{literal: "bold"}]}, %MDEx.Text{literal: " text"}]}]}
iex> Enum.at(doc, 1)
%MDEx.Paragraph{nodes: [%MDEx.Strong{nodes: [%MDEx.Text{literal: "bold"}]}, %MDEx.Text{literal: " text"}]}
iex> Enum.at(doc, 2)
%MDEx.Strong{nodes: [%MDEx.Text{literal: "bold"}]}
iex> Enum.at(doc, 3)
%MDEx.Text{literal: "bold"}
iex> Enum.at(doc, 4)
%MDEx.Text{literal: " text"}

Traverse and Update

You can also use the low-level MDEx.traverse_and_update/2 and MDEx.traverse_and_update/3 APIs to traverse each node of the AST and either update the nodes or do some calculation with an accumulator.

Streaming

Streaming ties together MDEx.new(streaming: true), MDEx.Document.put_markdown/3, and MDEx.Document.run/1 or MDEx.to_* so you can feed complete or incomplete Markdown fragments into the Document which will be completed on demand to render valid output.

Typical usage:

1. Start with MDEx.new(streaming: true) — the document enables streaming and buffers fragments.
2. Call MDEx.Document.put_markdown/3 as fragments arrive — the text is buffered and parsing/rendering is deferred.
3. Call MDEx.Document.run/1 or any MDEx.to_* — buffered fragments are parsed completing nodes to ensure valid output.

This is ideal for AI or chat apps where Markdown comes in bursts but must stay renderable.

For example, feeding **Fol produces a temporary MDEx.Strong node then adding low** replaces it with the final content on the next run.

iex> doc = MDEx.new(streaming: true) |> MDEx.Document.put_markdown("**Fol")
iex> MDEx.to_html!(doc)
"<p><strong>Fol</strong></p>"
iex> doc |> MDEx.Document.put_markdown("low**") |> MDEx.to_html!()
"<p><strong>Follow</strong></p>"

You can find a demo application in examples/streaming.exs.

Protocols

Enumerable

The Enumerable protocol allows us to call Enum functions to iterate over and manipulate the document tree. All enumeration follows the depth-first traversal order described above.

Count the nodes in a document:

iex> doc = ~MD"""
...> # Languages
...>
...> `elixir`
...>
...> `rust`
...> """
iex> Enum.count(doc)
7

Count how many nodes have the :literal attribute:

iex> doc = ~MD"""
...> # Languages
...>
...> `elixir`
...>
...> `rust`
...> """
iex> Enum.reduce(doc, 0, fn
...>   %{literal: _literal}, acc -> acc + 1
...>
...>   _node, acc -> acc
...> end)
3

Check if a node is member of the document:

iex> doc = ~MD"""
...> # Languages
...>
...> `elixir`
...>
...> `rust`
...> """
iex> Enum.member?(doc, %MDEx.Code{literal: "elixir", num_backticks: 1})
true

Map each node to its module name:

iex> doc = ~MD"""
...> # Languages
...>
...> `elixir`
...>
...> `rust`
...> """
iex> Enum.map(doc, fn %node{} -> inspect(node) end)
["MDEx.Document", "MDEx.Heading", "MDEx.Text", "MDEx.Paragraph", "MDEx.Code", "MDEx.Paragraph", "MDEx.Code"]

Collectable

The Collectable protocol allows you to build documents by collecting nodes or merging multiple documents together. This is particularly useful for programmatically constructing documents from various sources.

Merge two documents together using Enum.into/2:

iex> first_doc = ~MD[# First Document]
iex> second_doc = ~MD[# Second Document]
iex> Enum.into(second_doc, first_doc)
%MDEx.Document{
  nodes: [
    %MDEx.Heading{nodes: [%MDEx.Text{literal: "First Document"}], level: 1, setext: false},
    %MDEx.Heading{nodes: [%MDEx.Text{literal: "Second Document"}], level: 1, setext: false}
  ]
}

Collect individual nodes into a document:

iex> chunks = [
...>   %MDEx.Text{literal: "Hello "},
...>   %MDEx.Code{literal: "world", num_backticks: 1}
...> ]
iex> document = Enum.into(chunks, %MDEx.Document{})
%MDEx.Document{
  nodes: [
    %MDEx.Text{literal: "Hello "},
    %MDEx.Code{literal: "world", num_backticks: 1}
  ]
}
iex> MDEx.to_html!(document)
"Hello <code>world</code>"

Build a document incrementally by collecting mixed content:

iex> chunks = [
...>   %MDEx.Heading{nodes: [%MDEx.Text{literal: "Title"}], level: 1, setext: false},
...>   %MDEx.Paragraph{nodes: []},
...>   %MDEx.Text{literal: "Some text"},
...>   %MDEx.ListItem{nodes: [%MDEx.Text{literal: "Item 1"}]},
...>   %MDEx.Text{literal: " - WIP"},
...> ]
iex> document = Enum.into(chunks, %MDEx.Document{})
%MDEx.Document{
  nodes: [
    %MDEx.Heading{
      level: 1,
      nodes: [%MDEx.Text{literal: "Title"}],
      setext: false
    },
    %MDEx.Paragraph{
      nodes: [%MDEx.Text{literal: "Some text"}]
    },
    %MDEx.List{
      bullet_char: "-",
      delimiter: :period,
      is_task_list: false,
      list_type: :bullet,
      marker_offset: 0,
      nodes: [%MDEx.ListItem{nodes: [%MDEx.Text{literal: "Item 1 - WIP"}], list_type: :bullet, marker_offset: 0, padding: 2, start: 1, delimiter: :period, bullet_char: "-", tight: true, is_task_list: false}],
      padding: 2,
      start: 1,
      tight: true
    }
  ]
}
iex> MDEx.to_html!(document)
"<h1>Title</h1>\n<p>Some text</p>\n<ul>\n<li>Item 1 - WIP</li>\n</ul>"

Access

The Access behaviour gives you the ability to fetch and update nodes using different types of keys. Access operations also follow the depth-first traversal order when searching through nodes.

Access by Index

You can access nodes by their position in the depth-first traversal using integer indices:

iex> doc = ~MD[# Hello]
iex> doc[0]
%MDEx.Document{nodes: [%MDEx.Heading{nodes: [%MDEx.Text{literal: "Hello"}], level: 1, setext: false}]}
iex> doc[1]
%MDEx.Heading{nodes: [%MDEx.Text{literal: "Hello"}], level: 1, setext: false}
iex> doc[2]
%MDEx.Text{literal: "Hello"}

Negative indices access nodes from the end:

iex> doc = ~MD[# Hello **world**]
iex> doc[-1]  # Last node
%MDEx.Text{literal: "world"}

Access by Node Type

Starting with a simple Markdown document, let's fetch only the text node by matching the MDEx.Text node:

iex> ~MD[# Hello][%MDEx.Text{literal: "Hello"}]
[%MDEx.Text{literal: "Hello"}]

That's essentially the same as:

doc = %MDEx.Document{nodes: [%MDEx.Heading{nodes: [%MDEx.Text{literal: "Hello"}], level: 1, setext: false}]},

Enum.filter(
  doc,
  fn node -> node == %MDEx.Text{literal: "Hello"} end
)

The key can also be modules, atoms, and even functions! For example:

Fetch all Code nodes, either by MDEx.Code module or the :code atom representing the Code node:

iex> doc = ~MD"""
...> # Languages
...>
...> `elixir`
...>
...> `rust`
...> """
iex> doc[MDEx.Code]
[%MDEx.Code{num_backticks: 1, literal: "elixir"}, %MDEx.Code{num_backticks: 1, literal: "rust"}]
iex> doc[:code]
[%MDEx.Code{num_backticks: 1, literal: "elixir"}, %MDEx.Code{num_backticks: 1, literal: "rust"}]

Dynamically fetch Code nodes where the :literal (node content) starts with "eli" using a function to filter the result:

iex> doc = ~MD"""
...> # Languages
...>
...> `elixir`
...>
...> `rust`
...> """
iex> doc[fn node -> String.starts_with?(Map.get(node, :literal, ""), "eli") end]
[%MDEx.Code{num_backticks: 1, literal: "elixir"}]

That's the most flexible option, in case struct, modules, or atoms are not enough to match the node you want.

The Access protocol also allows us to update nodes that match a selector. In the example below we'll capitalize the content of all MDEx.Code nodes:

iex> doc = ~MD"""
...> # Languages
...>
...> `elixir`
...>
...> `rust`
...>
...> Continue...
...> """
iex> update_in(doc, [:document, Access.key!(:nodes), Access.all(), :code, Access.key!(:literal)], fn literal ->
...>   String.upcase(literal)
...> end)
%MDEx.Document{
  nodes: [
    %MDEx.Heading{nodes: [%MDEx.Text{literal: "Languages"}], level: 1, setext: false},
    %MDEx.Paragraph{nodes: [%MDEx.Code{num_backticks: 1, literal: "ELIXIR"}]},
    %MDEx.Paragraph{nodes: [%MDEx.Code{num_backticks: 1, literal: "RUST"}]},
    %MDEx.Paragraph{nodes: [%MDEx.Text{literal: "Continue..."}]}
  ]
}

String.Chars

Calling Kernel.to_string/1 will format it as CommonMark text:

iex> to_string(~MD[# Hello])
"# Hello"

Fragments (nodes without the parent %Document{}) are also formatted:

iex> to_string(%MDEx.Heading{nodes: [%MDEx.Text{literal: "Hello"}], level: 1})
"# Hello"

Inspect

The Inspect protocol provides two display formats for documents:

Tree format (default): Shows the document structure as a visual tree, making it easy to understand the hierarchy and relationships between nodes.

iex> ~MD[# Hello :smile:]
#MDEx.Document(3 nodes)<
└── 1 [heading] level: 1, setext: false
    ├── 2 [text] literal: "Hello "
    └── 3 [short_code] code: "smile", emoji: "😄"
>

Struct format: Shows the raw struct representation, useful for debugging and testing. To enable this format:

iex> Application.put_env(:mdex, :inspect_format, :struct)
iex> ~MD[# Hello :smile:]
%MDEx.Document{
  nodes: [
    %MDEx.Heading{
      nodes: [%MDEx.Text{literal: "Hello "}, %MDEx.ShortCode{code: "smile", emoji: "😄"}],
      level: 1,
      setext: false
    }
  ],
  # ... other fields
}

The struct format is particularly useful in tests where you need to see exact differences between expected and actual values. You can set this in your test/test_helper.exs:

Application.put_env(:mdex, :inspect_format, :struct)

Pipeline and Plugins

MDEx.Document is a Req-like API to transform Markdown documents through a series of steps in a pipeline.

Its main use case it to enable plugins, for example:

markdown = """
# Project Diagram

```mermaid
graph TD
    A[Enter Chart Definition] --> B(Preview)
    B --> C{decide}
    C --> D[Keep]
    C --> E[Edit Definition]
    E --> B
    D --> F[Save Image and Code]
    F --> B
```
"""

MDEx.new(markdown: markdown)
|> MDExMermaid.attach(mermaid_version: "11")
|> MDEx.to_html!()

To understand how it works, let's write that Mermaid plugin.

Writing Plugins

Let's start with a simple plugin as example to render Mermaid diagrams.

In order to render Mermaid diagrams, we need to inject a <script> into the document, as outlined in their docs:

<script type="module">
  import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs';
  mermaid.initialize({ startOnLoad: true });
</script>

Note that the package version is specified in the URL, so we'll add an option :mermaid_version to the plugin to let users specify the version they want to use.

By default, we'll use the latest version:

MDEx.new() |> MDExMermaid.attach()

But users can override it:

MDEx.new() |> MDExMermaid.attach(mermaid_version: "11")

Let's get into the actual code, with comments to explain each part:

defmodule MDExMermaid do
  alias MDEx.Document

  @latest_version "11"

  def attach(document, options \ []) do
    document
    # register option with prefix `:mermaid_` to avoid conflicts with other plugins
    |> Document.register_options([:mermaid_version])
    #  merge all options given by users
    |> Document.put_options(options)
    # actual steps to manipulate the document
    # see respective Document functions for more info
    |> Document.append_steps(enable_unsafe: &enable_unsafe/1)
    |> Document.append_steps(inject_script: &inject_script/1)
    |> Document.append_steps(update_code_blocks: &update_code_blocks/1)
  end

  # to render raw html and <script> tags
  defp enable_unsafe(document) do
    Document.put_render_options(document, unsafe: true)
  end

  defp inject_script(document) do
    version = Document.get_option(document, :mermaid_version, @latest_version)

    script_node =
      %MDEx.HtmlBlock{
        literal: """
        <script type="module">
          import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@#{version}/dist/mermaid.esm.min.mjs';
          mermaid.initialize({ startOnLoad: true });
        </script>
        """
      }

    Document.put_node_in_document_root(document, script_node)
  end

  defp update_code_blocks(document) do
    selector = fn
      %MDEx.CodeBlock{info: "mermaid"} -> true
      _ -> false
    end

    Document.update_nodes(
      document,
      selector,
      &%MDEx.HtmlBlock{literal: "<pre class="mermaid">#{&1.literal}</pre>", nodes: &1.nodes}
    )
  end
end

Now we can attach/1 that plugin into any MDEx document to render Mermaid diagrams.

Practical Examples

Here are some common patterns for working with MDEx documents that combine the protocols described above.

Update all code block nodes filtered by the selector function

Add line "// Modified" in Rust block codes:

iex> doc = ~MD"""
...> # Code Examples
...>
...> ```elixir
...> def hello do
...>   :world
...> end
...> ```
...>
...> ```rust
...> fn main() {
...>   println!("Hello");
...> }
...> ```
...> """
iex> selector = fn
...>   %MDEx.CodeBlock{info: "rust"} -> true
...>   _ -> false
...> end
iex> update_in(doc, [:document, Access.key!(:nodes), Access.all(), selector], fn node ->
...>   %{node | literal: "// Modified\n" <> node.literal}
...> end)
%MDEx.Document{
  nodes: [
    %MDEx.Heading{
      nodes: [%MDEx.Text{literal: "Code Examples"}],
      level: 1,
      setext: false
    },
    %MDEx.CodeBlock{
      info: "elixir",
      literal: "def hello do\n  :world\nend\n"
    },
    %MDEx.CodeBlock{
      info: "rust",
      literal: "// Modified\nfn main() {\n  println!(\"Hello\");\n}\n"
    }
  ]
}

Collect headings by level

iex> doc = ~MD"""
...> # Main Title
...>
...> ## Section 1
...>
...> ### Subsection
...>
...> ## Section 2
...> """
iex> Enum.reduce(doc, %{}, fn
...>   %MDEx.Heading{level: level, nodes: [%MDEx.Text{literal: text}]}, acc ->
...>     Map.update(acc, level, [text], &[text | &1])
...>   _node, acc -> acc
...> end)
%{
  1 => ["Main Title"],
  2 => ["Section 2", "Section 1"],
  3 => ["Subsection"]
}

Extract and transform task list items

iex> doc = ~MD"""
...> # Todo List
...>
...> - [ ] Buy groceries
...> - [x] Call mom
...> - [ ] Read book
...> """
iex> Enum.map(doc, fn
...>   %MDEx.TaskItem{checked: checked, nodes: [%MDEx.Paragraph{nodes: [%MDEx.Text{literal: text}]}]} ->
...>     {checked, text}
...>   _ -> nil
...> end)
...> |> Enum.reject(&is_nil/1)
[
  {false, "Buy groceries"},
  {true, "Call mom"},
  {false, "Read book"}
]

Bump all heading levels, except level 6

iex> doc = ~MD"""
...> # Main Title
...>
...> ## Subtitle
...>
...> ###### Notes
...> """
iex> selector = fn
...>   %MDEx.Heading{level: level} when level < 6 -> true
...>   _ -> false
...> end
iex> update_in(doc, [:document, Access.key!(:nodes), Access.all(), selector], fn node ->
...>   %{node | level: node.level + 1}
...> end)
%MDEx.Document{
  nodes: [
    %MDEx.Heading{nodes: [%MDEx.Text{literal: "Main Title"}], level: 2, setext: false},
    %MDEx.Heading{nodes: [%MDEx.Text{literal: "Subtitle"}], level: 3, setext: false},
    %MDEx.Heading{nodes: [%MDEx.Text{literal: "Notes"}], level: 6, setext: false}
  ]
}