Rag.Chunker.Paragraph (rag v0.3.4)

View Source

Paragraph-boundary chunking.

Splits on double newlines. Long paragraphs fall back to sentence splitting. Short paragraphs can be combined with min_chars.

Options

  • max_chars - Maximum characters per chunk (default: 500)
  • min_chars - Minimum characters, combines short paragraphs (default: nil)

Summary

Types

t()

Functions

chunk(chunker, text, opts)

Split text into paragraph-based chunks.

default_opts()

Returns default options for the paragraph chunker.

Types

t()

@type t() :: %Rag.Chunker.Paragraph{
  max_chars: pos_integer(),
  min_chars: pos_integer() | nil
}

Functions

chunk(chunker, text, opts)

@spec chunk(t(), String.t(), keyword()) :: [Rag.Chunker.Chunk.t()]

Split text into paragraph-based chunks.

default_opts()

@spec default_opts() :: keyword()

Returns default options for the paragraph chunker.