Tokenizer and Parser for RBNF rules.
Parse an RBNF rule definition
Scan and tokenize rule definition
Returns a list of rule subparts that can then be used for further processing or for turning into an AST for execution.
Using a leex lexer, tokenize a rule definition