PhoenixAI.Store.Memory.TokenCounter.Default (PhoenixAI.Store v0.1.0)

Copy Markdown View Source

Default token counter using a bytes / 4 heuristic.

This approximation is commonly used as a rough estimate for English text across most LLM tokenizers.