Rag.Chunker.Chunk (rag v0.3.4)

View Source

Represents a chunk of text with position information.

Fields

  • content - The text content of the chunk
  • start_byte - Byte offset where chunk begins in source text
  • end_byte - Byte offset where chunk ends in source text
  • index - Sequential index (0-based) among sibling chunks
  • metadata - Additional information (chunker type, hierarchy level, etc.)

Summary

Types

t()

Functions

extract_from_source(chunk, source_text)

Extract the chunk's content from the original text using byte positions.

new(attrs)

Create a new chunk from attributes.

valid?(chunk, source_text)

Check if chunk positions correctly match the content.

Types

t()

@type t() :: %Rag.Chunker.Chunk{
  content: String.t(),
  end_byte: non_neg_integer(),
  index: non_neg_integer(),
  metadata: map(),
  start_byte: non_neg_integer()
}

Functions

extract_from_source(chunk, source_text)

@spec extract_from_source(t(), String.t()) :: binary()

Extract the chunk's content from the original text using byte positions.

new(attrs)

@spec new(map()) :: t()

Create a new chunk from attributes.

valid?(chunk, source_text)

@spec valid?(t(), String.t()) :: boolean()

Check if chunk positions correctly match the content.