Rag.Chunker.Recursive (rag v0.3.4)

View Source

Hierarchical recursive chunking.

Tries paragraph boundaries first, falls back to sentence boundaries, then to character boundaries. Preserves semantic structure when possible.

Options

  • max_chars - Maximum characters per chunk (default: 500)
  • min_chars - Minimum characters per chunk (default: nil)

Metadata

Each chunk's metadata includes :hierarchy indicating the level at which it was split: :paragraph, :sentence, or :character.

Summary

Types

t()

Functions

chunk(chunker, text, opts)

Split text using hierarchical recursive chunking.

default_opts()

Returns default options for the recursive chunker.

Types

t()

@type t() :: %Rag.Chunker.Recursive{
  max_chars: pos_integer(),
  min_chars: pos_integer() | nil
}

Functions

chunk(chunker, text, opts)

@spec chunk(t(), String.t(), keyword()) :: [Rag.Chunker.Chunk.t()]

Split text using hierarchical recursive chunking.

default_opts()

@spec default_opts() :: keyword()

Returns default options for the recursive chunker.