GS1.Tokenizer.Base (gs1_barcode v0.1.2)

View Source

Compile–time base for GS1 tokenizers based on NimbleParsec. The tokenizer is deliberately “dumb”: it does not perform GS1 specific semantic checks or validations.

The tokenizer always returns AI prefixes in their minimal, two-digit form, even when the actual GS1 AI is longer.

All tokens produced by the tokenizer must undergo further normalization to reconstruct the canonical AI and produce Element Strings.

Summary

Types

token()

A single parsed token. Wraps the kind and the data tuple.

token_data()

The raw data tuple extracted by the tokenizer. Format: {ai_prefix, data_content}.

token_kind()

The type of the Application Identifier (AI).

tokenize_result()

The result returned by tokenize/2. Includes the list of tokens, remaining binary, context, line, and offset.

Types

token()

@type token() :: {token_kind(), token_data()}

A single parsed token. Wraps the kind and the data tuple.

token_data()

@type token_data() :: {String.t(), String.t()}

The raw data tuple extracted by the tokenizer. Format: {ai_prefix, data_content}.

token_kind()

@type token_kind() :: :ai_fixed | :ai_var

The type of the Application Identifier (AI).

  • :ai_fixed - The AI has a fixed length.
  • :ai_var - The AI has a variable length (delimited by FNC1/GS or end of string).

tokenize_result()

@type tokenize_result() ::
  {:ok, [token()], rest :: binary(), context :: map(), line :: pos_integer(),
   byte_offset :: pos_integer()}
  | {:error, reason :: String.t(), rest :: binary(), context :: map(),
     line :: pos_integer(), byte_offset :: pos_integer()}

The result returned by tokenize/2. Includes the list of tokens, remaining binary, context, line, and offset.