Helper functions for Layer 3 syntax normalization operations.
Contains utilities for character recognition, text processing, and common operations used across the syntax normalization layer.
Summary
Functions
Consume identifier characters from input starting at position.
Consume number characters from input starting at position.
Generic consume while predicate is true.
Consume whitespace from input and return {whitespace_string, new_position}.
Create a repair action record.
Check if a character can be part of an identifier.
Check if a character can start an identifier (letter, underscore, or UTF-8).
Check if a character can start an identifier in binary optimized mode.
Check if a string matches at a specific position (UTF-8 safe).
Check if a character is part of a number.
Safely pop from stack.
Check if a character is a UTF-8 letter (simplified approach).
Check if a token match is at a word boundary.
Functions
@spec consume_identifier(String.t(), non_neg_integer()) :: {String.t(), non_neg_integer()}
Consume identifier characters from input starting at position.
@spec consume_number(String.t(), non_neg_integer()) :: {String.t(), non_neg_integer()}
Consume number characters from input starting at position.
@spec consume_while(String.t(), non_neg_integer(), (String.t() -> boolean())) :: {String.t(), non_neg_integer()}
Generic consume while predicate is true.
@spec consume_whitespace(String.t(), non_neg_integer()) :: {String.t(), non_neg_integer()}
Consume whitespace from input and return {whitespace_string, new_position}.
@spec create_repair(String.t(), String.t(), non_neg_integer()) :: map()
Create a repair action record.
Check if a character can be part of an identifier.
Check if a character can start an identifier (letter, underscore, or UTF-8).
Check if a character can start an identifier in binary optimized mode.
@spec match_at_position?(String.t(), non_neg_integer(), String.t()) :: boolean()
Check if a string matches at a specific position (UTF-8 safe).
Check if a character is part of a number.
Safely pop from stack.
Check if a character is a UTF-8 letter (simplified approach).
@spec word_boundary?(String.t(), non_neg_integer(), String.t()) :: boolean()
Check if a token match is at a word boundary.