Cldr.Rbnf.Rule.tokenize

You're seeing just the function tokenize, go back to Cldr.Rbnf.Rule module for more information.

Scan and tokenize rule definition

Using a leex lexer, tokenize a rule definition