View Source API Reference TextChunker v0.3.1

Provides a high-level interface for text chunking, employing a configurable splitting strategy (defaults to recursive splitting). Manages options and coordinates the process, tracking chunk metadata.

Defines the Chunk struct, representing a contiguous block of text extracted during the splitting process. It stores the text content along with its corresponding byte range within the original input text.

Defines the contract that must be implemented for all text splitting strategies.

Handles recursive text splitting, aiming to adhere to configured size and overlap requirements. Employs a flexible separator-based approach to break down text into manageable chunks, while generating metadata for each produced chunk.

Handles separator configuration for the RecursiveChunk text chunking strategy.