The state used by the tokenizer.

%{ tokens: [], source: "...", position: 0, options: ... }

Also see

• Expug.TokenizerTools