View Source Tokenizers.Model (Tokenizers v0.3.2)
The struct and associated functions for the tokenizer model.
Link to this section Summary
Functions
Retrieves information about the model.
Link to this section Types
Link to this section Functions
Retrieves information about the model.
Information retrieved differs per model but all include model_type
.