CHANGELOG

v0.3.4 (2025-12-21)

Breaking Changes

• Chunking API redesigned: Replaced Rag.Chunking module with behavior-based Rag.Chunker system
  • Old: Rag.Chunking.chunk(text, strategy: :character, max_chars: 500)
  • New: Rag.Chunker.chunk(%Rag.Chunker.Character{max_chars: 500}, text)

Added

• Rag.Chunker behaviour for extensible chunking strategies
• Rag.Chunker.Chunk struct with byte position tracking (start_byte, end_byte)
• Built-in chunkers:
  • Rag.Chunker.Character - Fixed-size with smart boundaries
  • Rag.Chunker.Sentence - Sentence-boundary splitting
  • Rag.Chunker.Paragraph - Paragraph-boundary splitting
  • Rag.Chunker.Recursive - Hierarchical (paragraph -> sentence -> character)
  • Rag.Chunker.Semantic - Embedding-based similarity grouping
  • Rag.Chunker.FormatAware - Format-aware splitting via TextChunker
• Rag.Chunker.chunk_ingestion/3 for pipeline integration
• Rag.VectorStore.from_chunker_chunks/2 for VectorStore integration
• Byte-accurate position tracking enables source highlighting

Removed

• Rag.Chunking module (replaced by Rag.Chunker behavior)

Migration Guide

See docs/20251221/chunker-redesign/implementation.md for detailed migration instructions.

v0.3.3 (2025-12-21)

Fixes

• Fixed CI test failures when running without API keys by adding requires_llm_provider tag to tests that need auto-detected providers
• Removed unused alias warning in pipeline_test.exs

v0.3.2 (2025-12-21)

Fixes

• Extended conditional compilation to all Ecto-dependent modules:
  • Rag.Retriever.Semantic, Rag.Retriever.FullText, Rag.Retriever.Hybrid
  • Rag.Embedding.Service
• Eliminates all warnings when using rag_ex without Ecto

v0.3.1 (2025-12-21)

Fixes

• Made Ecto/pgvector dependencies optional - modules using Ecto.Schema and Ecto.Query are now conditionally compiled only when Ecto is available
• Projects can now use rag_ex without PostgreSQL/Ecto by omitting those dependencies

v0.3.0 (2025-12-21)

Major Features

• Modular RAG Architecture: Complete overhaul for configurable RAG pipelines
  • Rag.Retriever behaviour with Semantic, FullText, Hybrid, and Graph implementations
  • Rag.VectorStore.Store behaviour for pluggable vector backends
  • Rag.GraphStore behaviour for knowledge graph storage (pgvector and Neo4j ready)
  • Rag.Reranker behaviour with LLM-based reranking
• GraphRAG Support: Knowledge graph-enhanced retrieval
  • Entity and relationship extraction via LLM
  • Community detection with summarization
  • Local, global, and hybrid graph search modes
  • PostgreSQL-based graph storage with recursive CTEs
• Advanced Chunking Strategies: Flexible text splitting
  • Character-based chunking with smart boundaries
  • Sentence-based chunking
  • Semantic chunking with embedding similarity
  • Recursive chunking for hierarchical documents
• Pipeline System: Composable RAG pipelines
  • Rag.Pipeline struct for pipeline definitions
  • Rag.Pipeline.Executor for step orchestration
  • Parallel step execution with caching
  • Error handling and retry logic
• Reranking: Improve retrieval quality
  • LLM-based reranking with configurable prompts
  • Cross-encoder style relevance scoring
  • Cohere reranker integration ready
• Multi-LLM Provider Support: Add Rag.Ai.Gemini, Rag.Ai.Claude, and Rag.Ai.Codex providers for Gemini, Claude, and OpenAI-compatible APIs
• Smart Router: New Rag.Router module with pluggable routing strategies:
  • Rag.Router.Fallback - Try providers in order until one succeeds
  • Rag.Router.RoundRobin - Distribute load across providers
  • Rag.Router.Specialist - Route based on task type
• Vector Store: New Rag.VectorStore module with pgvector integration:
  • Semantic search with embeddings
  • Full-text search with PostgreSQL tsvector
  • Hybrid search with RRF (Reciprocal Rank Fusion) scoring
  • Text chunking with overlap support
  • Rag.VectorStore.Chunk Ecto schema for document storage
• Embedding Service: New Rag.Embedding.Service GenServer for managed embedding operations with auto-batching
• Agent Framework: New Rag.Agent module for building tool-using agents:
  • Rag.Agent.Session for conversation memory management
  • Rag.Agent.Registry for tool registration
  • Rag.Agent.Tool behaviour for custom tools
• Built-in Agent Tools:
  • Rag.Agent.Tools.SearchRepos - Semantic search over indexed repositories
  • Rag.Agent.Tools.ReadFile - Read file contents with optional line ranges
  • Rag.Agent.Tools.GetRepoContext - Get repository structure and metadata
  • Rag.Agent.Tools.AnalyzeCode - Parse and analyze code structure
• Provider Capabilities: New Rag.Ai.Capabilities module to check provider feature support

Enhancements

• New live examples for all major features
• Comprehensive test coverage with supertester principles
• Updated documentation with modular RAG guides

Breaking Changes

• Removed igniter-based Mix tasks: rag.install, rag.gen_rag_module, rag.gen_servings, rag.gen_eval
• Library no longer includes its own Ecto Repo - consuming applications must provide their own

Dependencies

• Added pgvector, ecto_sql, postgrex for vector store functionality
• Temporarily disabled torus and igniter due to Elixir 1.18 compatibility issues with inflex

v0.2.3

• Add Rag.Ai.Ollama as ollama provider
• Add build_context/3, build_context_sources/3, build_prompt/3 to enable full pipeline interface
• Enable streaming of responses (@W3NDO, thank you for your contribution!)

v0.2.2

• Add ref to Generation for referencing in telemetry handler

v0.2.1

• fix credo issues in generator
• helpful error for missing servings in nx provider
• fix typespecs with optional fields
• more robust fulltext search with postgres in generator

v0.2.0

• syntax updates
• unified embedding, generation, and evaluation modules

v0.1.0

• initial release