# `Color.CSS.Tokenizer`

A small, paren-aware tokenizer for the bodies of CSS color
functions (`rgb(...)`, `hsl(...)`, `lab(...)`, `color(...)`,
`device-cmyk(...)`, etc).

The tokenizer walks the input character-by-character, splits at
whitespace and commas at depth zero, and groups any nested
function call (e.g. `calc(255 / 2)`) into a single
`{:func, name, body}` token. This is enough structure for
`Color.CSS` to dispatch each component to the right resolver
without re-parsing the surrounding context.

## Token shapes

* `{:number, float}` — a bare number, e.g. `255`, `0.5`, `-1.2`.
* `{:percent, float}` — a percentage value (the raw percent
  number, *not* divided by 100), e.g. `{:percent, 50.0}` for `50%`.
* `{:hue, float, unit}` — an angle with an explicit unit
  (`:deg`, `:rad`, `:grad`, `:turn`).
* `:none` — the literal `none` keyword.
* `{:ident, name}` — a bare identifier (`from`, `in`, `red`,
  `display-p3`, or a relative-color component reference).
* `{:func, name, body}` — a parenthesised function call,
  name lower-cased, body retained as a string for the consumer
  to parse.
* `{:hex, hex}` — `#rrggbb` etc, with the leading `#` stripped.
* `{:slash}` — the `/` separator that introduces an alpha value.

# `tokenize`

Tokenizes a CSS color function body.

### Arguments

* `string` is the body of a CSS function — everything between
  its outermost `(` and `)`.

### Returns

* `{:ok, [token]}` on success.

* `{:error, reason}` on syntax errors (currently only unbalanced
  parens).

### Examples

    iex> Color.CSS.Tokenizer.tokenize("255 0 0")
    {:ok, [{:number, 255.0}, {:number, 0.0}, {:number, 0.0}]}

    iex> Color.CSS.Tokenizer.tokenize("none 0 0")
    {:ok, [:none, {:number, 0.0}, {:number, 0.0}]}

    iex> Color.CSS.Tokenizer.tokenize("calc(255 / 2) 0 0")
    {:ok, [{:func, "calc", "255 / 2"}, {:number, 0.0}, {:number, 0.0}]}

    iex> Color.CSS.Tokenizer.tokenize("from red r g b")
    {:ok,
     [{:ident, "from"}, {:ident, "red"}, {:ident, "r"}, {:ident, "g"}, {:ident, "b"}]}

    iex> Color.CSS.Tokenizer.tokenize("255 0 0 / 50%")
    {:ok, [{:number, 255.0}, {:number, 0.0}, {:number, 0.0}, {:slash}, {:percent, 50.0}]}

    iex> Color.CSS.Tokenizer.tokenize("display-p3 1 0 0")
    {:ok, [{:ident, "display-p3"}, {:number, 1.0}, {:number, 0.0}, {:number, 0.0}]}

---

*Consult [api-reference.md](api-reference.md) for complete listing*