Struct representing a text chunk with metadata.
Fields
:text- The text content of the chunk:start_byte- The starting index of the chunk in the original text:end_byte- The ending index of the chunk in the original text:token_count- The number of tokens in the chunk:embedding- The embedding vector for the chunk (optional)
Summary
Types
@type t() :: %Chunx.Chunk{ embedding: Nx.Tensor.t() | nil, end_byte: non_neg_integer(), start_byte: non_neg_integer(), text: String.t(), token_count: pos_integer() }
Functions
@spec new( String.t(), non_neg_integer(), non_neg_integer(), pos_integer(), Nx.Tensor.t() | nil ) :: t()