# `Localize.Message.Highlighter`
[🔗](https://github.com/elixir-localize/localize/blob/v0.25.0/lib/localize/message/highlighter.ex#L1)

Produces a classified token stream from an MF2 AST for syntax
highlighting.

The output is a flat list of `{class :: atom, text :: String.t()}`
tuples. Each tuple represents one syntactic region of the canonical
message — punctuation, a variable reference, a function name, a
literal, plain text, etc. Formatters (`Localize.Message.Formatter.HTML`,
`Localize.Message.Formatter.ANSI`, `Localize.Message.Formatter.Plain`)
consume the token stream and produce rendered output.

Concatenating every token's text field yields the canonical MF2
message (the same output as `Localize.Message.Print.to_string/1`) —
the `Plain` formatter does exactly that. This property is verified
by the test suite.

### Token classes

Classes follow the same taxonomy as tree-sitter highlight captures,
so one stylesheet can style both server-rendered HTML and the
browser-side [`mf2_wasm_editor`](https://hex.pm/packages/mf2_wasm_editor).
Atoms carry underscores; the HTML formatter converts them to hyphens
on emission (so `:punctuation_bracket` becomes `.mf2-punctuation-bracket`).

* `:punctuation_bracket` — `{`, `}`, `{{`, `}}`, `|`, `=`, `/`,
  whitespace inside expressions, and similar structural characters.

* `:variable` — `$name` (entire token including `$`).

* `:function` — `:number`, `:date`, custom function names (entire
  token including `:`).

* `:keyword` — `.input`, `.local`, `.match` keywords.

* `:tag` — markup tag names (`bold` in `{#bold}`).

* `:attribute` — attribute names (`@translate`).

* `:property` — option names (`style` in `style=|short|`).

* `:string` — quoted literal content including the `|` delimiters.

* `:number` — numeric literals (integer and float).

* `:text` — pattern text (plain message content).

* `:string_escape` — `\{`, `\|`, `\\` escape sequences.

* `:constant_builtin` — `*` (catchall variant key).

# `class`

```elixir
@type class() ::
  :punctuation_bracket
  | :variable
  | :function
  | :keyword
  | :tag
  | :attribute
  | :property
  | :string
  | :number
  | :text
  | :string_escape
  | :constant_builtin
```

# `token`

```elixir
@type token() :: {class(), String.t()}
```

# `to_tokens`

```elixir
@spec to_tokens(list() | tuple()) :: [token()]
```

Walks the AST and returns a list of classified tokens.

### Arguments

* `ast` is a parsed MF2 AST as returned by
  `Localize.Message.Parser.parse/1`.

### Returns

* A list of `{class, text}` tuples.

---

*Consult [api-reference.md](api-reference.md) for complete listing*