Changelog

View Source

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.7.4] - 2025-12-30

Added

  • CI testing against Weaviate versions 1.31.x through 1.35.x (9 total versions)
  • Code coverage reporting via Codecov with excoveralls
  • Coverage badge in README
  • Supported Weaviate versions table in README

Multi-Target Reference Improvements

  • WeaviateEx.Types.Beacon module for parsing and building beacon URLs
    • Beacon.parse/1 - Extract collection and UUID from beacon URLs
    • Beacon.build/2 - Build beacon URLs from UUID and optional collection
    • Beacon.to_map/2 - Create beacon maps for API requests
    • Beacon.extract_uuid/1, Beacon.extract_collection/1 - Extract components
    • Beacon.valid?/1 - Validate beacon URL format
  • References.add/5, delete/5 now accept ReferenceToMulti struct directly
  • References.replace/5 now accepts list of ReferenceToMulti structs
  • Reference.from_beacon/1 and Reference.from_map/1 now extract target collection from beacon
  • Improved multi-target reference support in all reference operations

gRPC Reranking

  • gRPC-based reranking for search queries via Query.rerank/2
  • Rerank.to_grpc/1 for converting rerank config to gRPC Rerank message
  • Rerank score parsing in gRPC result metadata (rerankScore)
  • Full support with near_text, near_vector, near_object, hybrid, and bm25 queries
  • Works with all reranker modules: Cohere, Transformers, VoyageAI, JinaAI, NVIDIA, ContextualAI

Aggregate Variants (Near Object, Hybrid, Near Image)

  • Aggregate.with_near_object/4 for similarity-based aggregation by reference object UUID
    • Supports :distance and :certainty threshold parameters
    • Full property metrics support (numeric, text, boolean)
  • Aggregate.with_hybrid/4 for hybrid (keyword + vector) search aggregation
    • :alpha parameter to control keyword/vector weighting (0.0-1.0, default 0.5)
    • :fusion_type option (:ranked, :relative_score)
    • Full property metrics support
  • Aggregate.with_near_image/4 for image similarity aggregation
    • Supports :distance, :certainty, and :target_vectors parameters
    • For multi-modal collections with image vectorizers
  • WeaviateEx.Aggregate.Metrics helper module for cleaner metric specifications
    • Metrics.count/0 - Meta count metric
    • Metrics.text/2 - Text property metrics (top_occurrences, count)
    • Metrics.number/2 - Numeric property metrics (sum, mean, median, mode, min, max)
    • Metrics.integer/2 - Integer property metrics (same as number)
    • Metrics.boolean/2 - Boolean property metrics (percentage, totals)
    • Metrics.date/2 - Date property metrics (min, max, median, mode)

Fluent Tenant-Scoped API

  • WeaviateEx.TenantCollection for fluent tenant-scoped operations
  • Collections.with_tenant/3 returns tenant-scoped collection reference
  • TenantCollection.insert/3, insert_many/3, get/3, update/4, delete/3
  • TenantCollection.query/1 returns tenant-scoped query builder
  • TenantCollection.batch/2 starts tenant-scoped batch context
  • Query.with_tenant/2 for explicit tenant scoping (alias for Query.tenant/2)

Multi-Modal Search Convenience Methods

  • Query.with_near_image/3 for image similarity search with automatic input handling
  • Query.with_near_audio/3 for audio similarity search
  • Query.with_near_video/3 for video similarity search
  • Query.with_near_thermal/3 for thermal image search
  • Query.with_near_depth/3 for depth map search
  • Query.with_near_imu/3 for IMU data search
  • Query.with_near_media/4 for generic multi-modal search with type parameter
  • WeaviateEx.Types.MediaInput for flexible media input handling (file paths, base64, raw binary)
  • WeaviateEx.Types.MediaType for media type validation

Object TTL Enhancements

  • ObjectTTL.from_duration/1 - Human-readable TTL configuration (e.g., from_duration(hours: 24))
  • Collections.update_ttl/3 - Convenience function for updating TTL on existing collections
  • Extended README documentation with comprehensive Object TTL examples

WCS Compatibility (Weaviate Cloud Service)

  • Automatic X-Weaviate-Cluster-URL header for WCS connections
    • Required for Weaviate embedding service integration
    • Auto-added by Connect.to_weaviate_cloud/1
  • Proper gRPC host parsing for .weaviate.network clusters
    • Uses {ident}.grpc.{domain} pattern (e.g., my-cluster.grpc.weaviate.network)
    • Other WCS domains use grpc-{host} pattern
  • WeaviateEx.Error.VersionError exception for clear version mismatch errors
    • Includes workaround instructions for skip_init_checks: true
  • Server version compatibility check on connect (requires 1.27.0+)
  • gRPC availability check on connect
  • skip_init_checks option in Connect.to_weaviate_cloud/1 to bypass validation
  • Extended README documentation with WCS connection examples

Developer Tooling

  • Benchmark suite with Benchee (mix weaviate.bench)
  • Batch and query performance benchmarks in bench/
  • Pre-commit hooks configuration (.pre-commit-config.yaml)
  • Profiling guide (guides/profiling.md)
  • Mix.Tasks.Weaviate.Bench task for running benchmarks

gRPC Generative Search (RAG)

  • gRPC Generative Module (WeaviateEx.GRPC.Generative):

    • build/1 - Build GenerativeSearch message from simple config
    • build_with_provider/1 - Build with provider-specific configuration
    • parse_generative_result/1 - Parse grouped results from gRPC reply
    • extract_single_results/1 - Extract per-object generations from results
    • Support for all major providers: OpenAI, Anthropic, Cohere, Mistral, Ollama, Google, AWS, Databricks, FriendliAI, NVIDIA, XAI, ContextualAI, Anyscale
    • Provider-specific parameters: model, temperature, max_tokens, top_p, top_k, etc.
    • OpenAI reasoning model support (verbosity, reasoning_effort for o1/o3)
  • gRPC Search with Generative (WeaviateEx.GRPC.Services.Search):

    • New generative option for all search request builders
    • Single prompt generation via single_prompt config
    • Grouped task generation via grouped_task and grouped_properties
    • Works with near_text, near_vector, hybrid, bm25 searches
    • ~2-3x lower latency compared to GraphQL generative
  • GenerativeResult Enhancements (WeaviateEx.Query.GenerativeResult):

    • from_grpc_response/1 - Parse gRPC SearchReply into typed result
    • Extended struct with :metadata field for usage/debug info
    • Proper parsing of grouped and per-object generative results
    • Vector parsing from gRPC binary format

HTTP Transport-Level Retry

  • HTTP Retry Module (WeaviateEx.Protocol.HTTP.Retry):
    • with_retry/2 - Execute function with automatic retry on transient errors
    • retryable_status_code?/1 - Check if HTTP status code is retryable
    • retryable_status_codes/0 - Get list of retryable status codes
    • default_retry_opts/0 - Get default retry options
    • Retryable status codes: 408, 429, 500, 502, 503, 504
    • Exponential backoff with jitter (±10%)
    • Configurable: max_retries (default: 3), base_delay_ms (default: 100), max_delay_ms (default: 5000)

Per-Operation Timeout Support

  • HTTP Timeout Module (WeaviateEx.Protocol.HTTP.Timeout):
    • for_operation/2 - Get timeout for specific operation type
    • batch_multiplier/0 - Get batch timeout multiplier (10×)
    • operation_category/1 - Categorize operation for timeout selection
    • Operation types: :query, :insert, :batch, :init
    • Batch operations automatically use extended timeout (insert × 10)

HTTP Client Enhancements

  • HTTP client now retries on transient HTTP status codes (503, 429, etc.)
  • Automatic batch timeout detection for /batch endpoints
  • Support for operation option to specify operation type
  • Support for retry config in client options
  • Merged retry options from client config and per-request options

Batch Safety Features

  • Batch Config Module (WeaviateEx.Batch.Config):

    • Centralized configuration for batch safety options
    • new/1 - Create config with custom options
    • auto_retry_enabled?/1 - Check if auto-retry is enabled
    • merge/2 - Merge two configurations
    • to_keyword/1 - Convert config to keyword list
    • Defaults: max_stored_results: 100_000, auto_retry: true, max_retries: 3, retry_delay_ms: 1000
  • Retry Queue Module (WeaviateEx.Batch.RetryQueue):

    • GenServer for automatic re-queuing of failed batch objects
    • enqueue_failed/2 - Add failed objects to retry queue
    • drain/1 - Get all queued objects for manual processing
    • get_retry_count/2 - Get retry count for specific UUID
    • clear/1 - Clear queue and reset retry counts
    • calculate_backoff/2 - Exponential backoff calculation
    • retryable_error?/1 - Check if error is retryable
    • Supports permanent failure callback for objects exceeding max_retries
    • Jitter (±20%) for backoff to prevent thundering herd
  • Dynamic Batch Auto-Retry (WeaviateEx.Batch.Dynamic):

    • New options: :auto_retry, :max_retries, :retry_delay_ms, :on_permanent_failure
    • Automatically re-queues failed objects with retryable errors
    • Integrates with RetryQueue for retry tracking
    • Retryable errors: rate limits, UNAVAILABLE, RESOURCE_EXHAUSTED, ABORTED, DEADLINE_EXCEEDED

Journey Tests (Phoenix/Plug Integration)

  • Journey Test Scenarios (WeaviateEx.Journey.Scenarios):
    • simple/1 - Create collection, insert 100 objects, query, and cleanup
    • batch_insert_and_search/1 - Batch insert 1000 objects and perform vector search
    • concurrent_operations/1 - Handle 10 concurrent operations without errors
  • Phoenix Integration Tests (test/journey/phoenix_test.exs):
    • Client initialized at startup survives multiple requests
    • Concurrent requests share client safely
    • Client properly cleaned up on shutdown
    • Full journey scenarios executed through Phoenix endpoints
  • Plug Integration Tests (test/journey/plug_test.exs):
    • Client accessible via Application env
    • Journey scenarios work through Plug router
    • Concurrent Plug requests share client safely
  • Test Configuration:
    • Journey tests excluded by default (run with --include journey)
    • Added :journey tag to ExUnit exclusions

Property Value Serialization

  • Serializable Protocol (WeaviateEx.Types.Serializable):

    • Protocol for serializing Elixir types to Weaviate-compatible values
    • Automatic serialization when creating/updating objects
    • Implementations for: DateTime, NaiveDateTime, Date, GeoCoordinate, PhoneNumber, Blob
    • Extensible - implement protocol for custom types
  • Date Type Serialization:

    • Date structs serialize to ISO8601 at midnight UTC (e.g., ~D[2024-01-01]"2024-01-01T00:00:00Z")
  • Blob Struct (WeaviateEx.Types.Blob):

    • New struct for wrapping binary data: Blob.new(binary_data)
    • Blob.from_file/1 / Blob.from_file!/1 - Create from file
    • Blob.to_base64/1 - Convert to base64 string
    • Blob.from_base64/1 - Create from base64 string
    • Automatic base64 encoding when used in object properties
  • Deserialize Module (WeaviateEx.Types.Deserialize):

    • deserialize/2 - Parse Weaviate response values with type hints
    • deserialize!/2 - Same but raises on error
    • deserialize_properties/2 - Deserialize map with schema hints
    • auto_deserialize/1 - Auto-detect types based on value structure
    • Supported types: :date, :geo_coordinates, :phone_number, :blob, :auto

Changed

  • Default local test Weaviate version updated to 1.35.0
  • Unit tests now generate coverage reports uploaded to Codecov
  • WeaviateEx.Objects.Payload now uses Serializable protocol internally for type serialization
  • Payload documentation updated to reflect all supported serialization types

[0.7.3] - 2025-12-29

Added

Transport & Auth

  • WCS gRPC Host Parsing: .weaviate.network clusters now use ident.grpc.<domain> (matches Python client)
  • Proxy + Pool Wiring: client connection/proxy options are applied to Finch pools and gRPC channels
  • OIDC Client Auth: Client.connect/1 accepts WeaviateEx.Auth configs and auto-refreshes tokens
  • Skip Init Checks: new :skip_init_checks option to bypass meta/version/gRPC health checks
  • gRPC Header Forwarding: additional_headers are included in gRPC metadata by default

Batch Operations

  • Wait for Vector Indexing (WeaviateEx.Batch.VectorIndexing):
    • wait_for_indexing/3 - Wait for all vectors to be indexed after batch operations
    • wait_for_shards/3 - Wait for specific shards to have vectors indexed
    • shard_ready?/3 - Check if a specific shard is ready for queries
    • get_queue_status/2 - Get vector queue status for all shards
    • total_pending_vectors/2 - Get total pending vector count
    • all_indexed?/2 - Check if all vectors are indexed
    • Options: :timeout, :poll_interval, :how_many_failures, :tenant
    • Exposed via WeaviateEx.API.Batch.wait_for_vector_indexing/3
  • All-Fail Batch Errors (WeaviateEx.API.Batch, WeaviateEx.Batch):
    • Returns :batch_all_failed error when every object in a batch fails
  • gRPC Stream Backoff Handling (WeaviateEx.Batch.Stream):
    • Applies server backoff by updating the stream buffer size

RBAC Enhancements

  • User Assignments with Type (WeaviateEx.API.RBAC):

    • get_user_assignments/3 - Get users assigned to role with type info
    • UserAssignment struct with user_id and user_type fields
    • Supports :db_user, :db_env_user, :oidc user types
  • Group Assignments with Type (WeaviateEx.API.RBAC):

    • get_group_assignments/3 - Get groups assigned to role with type info
    • GroupAssignment struct with group_id and group_type fields
  • Nodes Permission Collection Filter (WeaviateEx.RBAC.Permissions):

    • nodes/2 now supports :collection option for verbose mode
    • Example: Permissions.nodes(:verbose, collection: "Article")

Collections/Schema

  • Reconfigure Module (WeaviateEx.Reconfigure):
    • inverted_index/1 - Build inverted index update config (BM25, cleanup, stopwords)
    • replication/1 - Build replication update config (factor, async, deletion strategy)
    • vector_index_hnsw/1 - Build HNSW index update config (ef, max_connections, etc.)
    • named_vectors_update/2 - Build named vector update config
    • multi_tenancy/1 - Build multi-tenancy update config
    • description/1 - Build description update
    • merge/1 - Combine multiple update configs
  • Property Range Filters (WeaviateEx.Property):
    • index_range_filters option maps to indexRangeFilters on schema properties
  • Config Struct Serialization (WeaviateEx.Collections, WeaviateEx.API.Collections):
    • Auto-serializes ObjectTTL, AutoTenant, and MultiTenancyConfig in create/update payloads
  • Collection Handle Defaults (WeaviateEx.Collection):
    • new/3, with_tenant/2, with_consistency/2 and data ops apply default tenant/consistency

Query Enhancements

  • Multi-Vector Queries (WeaviateEx.Query.NearVector):
    • single/2 - Single vector query with options
    • list_of_vectors/2 - Query with multiple vectors over single space
    • per_target/2 - Query with different vectors per named vector space
    • weighted_targets/2 - Weighted multi-target vector query
    • to_api/1 - Convert to API format
  • gRPC Search Parity (WeaviateEx.GRPC.Services.Search):
    • gRPC search now includes filters, group_by, target vectors, near_image/near_media, references, and vector metadata
    • gRPC parsing now returns reference properties and vector metadata when requested

Data Operations

  • Fetch Objects by IDs (WeaviateEx.API.Data):
    • fetch_objects_by_ids/4 helper with input order preserved
  • Fetch Objects by IDs (WeaviateEx.Objects):
    • fetch_objects_by_ids/3 convenience wrapper using global config
  • Payload Validation (WeaviateEx.Objects.Payload):
    • Client-side checks for required properties and reserved names (id, vector)

Generative AI

Aggregation

  • Near Image Aggregation (WeaviateEx.API.Aggregate):
    • with_near_image/4 - Aggregate with near_image similarity constraint
    • Support for :certainty, :distance, :target_vectors options

Users Management

  • Include Permissions Parameter (WeaviateEx.API.Users.DB):

    • get_roles/3 now supports :include_permissions option
    • When true, returns roles with full permission details
  • Deactivate with Key Revocation (WeaviateEx.API.Users.DB):

    • deactivate/3 now supports :revoke_key option
    • When true, also revokes the user's API key

Vectorizers

  • Multi-Vector Encoding (WeaviateEx.API.MultiVector.Encoding):
    • muvera/1 - Configure Muvera encoding for ColBERT-style multi-vectors
    • Options: :ksim, :dprojections, :repetitions
    • none/0 - Raw multi-vectors without encoding

Tenant Statuses

  • Transitional Tenant Statuses (WeaviateEx.Types.Tenant):
    • Added :offloading status - Tenant is being offloaded to cold storage
    • Added :onloading status - Tenant is being loaded from cold storage
    • Full type support in activity_status type

Cluster API Enhancements

  • Shard Name Filtering (WeaviateEx.API.Cluster):

    • nodes/2 now supports :shard option to filter nodes by specific shard name
    • Combined filtering: Cluster.nodes(client, collection: "Article", shard: "shard-0")
  • Delete All Replications (WeaviateEx.API.Cluster):

    • delete_all_replications/1 - Batch delete all completed, failed, and cancelled replication records
  • Query Sharding State (WeaviateEx.API.Cluster):

    • query_sharding_state/3 - Query sharding state to see which shards exist and their replica nodes
    • :shard option for filtering by specific shard
    • Returns ShardingState struct with collection name and list of ShardReplicas
  • ShardingState Module (WeaviateEx.Cluster.ShardingState):

    • ShardingState struct for collection sharding information
    • ShardReplicas submodule for shard-to-node replica mappings
    • from_api/1 - Parse sharding state from API response

Shard Struct Enhancements

  • New Shard Fields (WeaviateEx.Cluster.Shard):

    • node field - Which node hosts this shard
    • loaded field - Boolean indicating if shard is loaded into memory (for lazy loading scenarios)
  • LAZY_LOADING Status (WeaviateEx.Cluster.Shard):

    • Added :lazy_loading to shard status enum
    • parse_status/1 and status_to_api/1 support for "LAZY_LOADING" status

Vectorizer Enhancements

  • Text2VecOpenAI Vectorizer (WeaviateEx.API.Vectorizers.Text2VecOpenAI):

    • Full OpenAI vectorizer configuration support
    • Options: :model, :dimensions, :type, :base_url, :vectorize_collection_name
    • to_api/1, from_api/1 for API serialization
  • Text2VecCohere Vectorizer (WeaviateEx.API.Vectorizers.Text2VecCohere):

    • Full Cohere vectorizer configuration support
    • Options: :model, :dimensions, :truncate, :base_url, :vectorize_collection_name
    • to_api/1, from_api/1 for API serialization
  • New Rerankers (WeaviateEx.API.RerankerConfig):

    • nvidia/2 - NVIDIA NIM reranker configuration
    • contextualai/2 - ContextualAI reranker configuration

Query Enhancements

  • Hybrid Search Improvements (WeaviateEx.Query):
    • :max_vector_distance option for filtering results by vector distance threshold
    • :bm25_operator option for customizing BM25 matching behavior (AND/OR)
    • BM25Operator struct for configuring operator and minimum match count

Data Operations

  • Fetch Objects by IDs (WeaviateEx.API.Data):

    • fetch_objects_by_ids/4 - Efficiently fetch multiple objects by their IDs
    • Uses GraphQL ContainsAny filter for optimal performance
    • Supports :return_properties and :tenant options
  • Iterator References (WeaviateEx.Iterator):

    • :return_references option for including cross-references in iteration
    • Support for simple references and references with specific properties
    • Automatic class name inference from reference names
  • Insert Many Convenience (WeaviateEx.Collections):

    • insert_many/3 - Batch insert objects with automatic formatting
    • Supports both raw property maps and structured objects
    • Automatic ID, vector, and tenant handling

Multi-Tenancy Enhancements

  • AutoTenant Configuration (WeaviateEx.Config.AutoTenant):
    • enable/1 - Enable automatic tenant creation
    • :auto_delete_timeout option for automatic cleanup of empty tenants
    • disable/0, to_map/1, from_map/1 for full lifecycle management

RBAC Enhancements

  • Role Scope Parameter (WeaviateEx.RBAC.Permissions):
    • :scope option in roles/3 for permission scope control
    • Supports :match (exact) and :all (wildcard) scopes

Property Enhancements

  • Multi-Target References (WeaviateEx.Property):
    • reference/3 now accepts a list of target collections
    • multi_reference/3 alias for explicit multi-target references
    • Example: Property.reference("hasContent", ["Article", "BlogPost"])

Integration Enhancements

  • Rate Limit Headers (WeaviateEx.Integrations):
    • OpenAI: :requests_per_minute_embeddings, :tokens_per_minute_embeddings
    • Cohere: :requests_per_minute_embeddings
    • Custom :base_url support for both providers

Auth Enhancements

  • OIDC Grant Type Validation (WeaviateEx.Auth.TokenManager):
    • Validation of OIDC grant types on startup
    • Supported types: :oidc_client_credentials, :oidc_password
    • Clear error messages for invalid or missing auth configuration

Fixed

Debug

Changed

  • Updated @valid_statuses in Tenant to include transitional statuses
  • Improved alias ordering in multiple modules for consistency
  • Query.execute/3 now falls back to GraphQL when a query uses options not supported by gRPC (rerank, sorting, cursor pagination, hybrid vectors)

Stats

  • 2612 tests passing (138 new tests added)
  • Full Python client parity for gap closure features
  • Zero warnings, errors, dialyzer issues, or credo violations

[0.7.2] - 2025-12-29

Added

Multi-Tenancy Enhancements

  • Fluent TenantClient API (WeaviateEx.TenantClient):

    • with_tenant/2 - Create tenant-scoped client wrapper
    • collection/2 - Set collection for subsequent operations
    • insert/3, get/3, update/4, replace/4, delete/3 - CRUD operations
    • query/2, near_vector/3, near_text/3, hybrid/3, bm25/3 - Search operations
    • batch_insert/3, batch_delete/3 - Batch operations
    • count/2, exists?/2 - Utility operations
    • Automatic tenant injection into all operations
  • Batch Tenant Updates (WeaviateEx.API.Tenants):

    • batch_update/3 - Update tenants in batches of 100 (matching Python client)
    • batch_size/0 - Returns configured batch size
  • Tenant Struct (WeaviateEx.Types.Tenant):

    • Full typed activity status support: :active, :inactive, :hot, :cold, :frozen, :offloaded
    • new/2, to_map/1, from_map/1 - Construction and serialization
    • set_status/2, active?/1, inactive?/1 - Status management
    • valid_status?/1, valid_statuses/0 - Validation helpers

Backup/Restore Enhancements

  • Backup List Sorting (WeaviateEx.API.Backup):

    • sort_by_starting_time_asc option in list/3 for chronological ordering
  • Cancel with Dynamic Location (WeaviateEx.API.Backup):

    • :location option in cancel/4 for dynamic backup location configuration
  • BackupInfo Metadata (WeaviateEx.Backup.Status.BackupInfo):

    • started_at - When the backup started (DateTime)
    • completed_at - When the backup completed (DateTime)
    • size_bytes - Total backup size in bytes
    • error - Error message if backup failed

Auth/Connection Improvements

  • WCS Header Auto-Detection (WeaviateEx.Client.Config):

    • is_wcs_host?/1 - Detect Weaviate Cloud Services instances
    • maybe_add_wcs_headers/1 - Add WCS-specific headers automatically
    • new_with_wcs_detection/1 - Create config with automatic WCS detection
    • Detects known WCS domains: weaviate.network, wcs.api.weaviate.io, semi.network, weaviate.cloud
  • Server Version Compatibility (WeaviateEx.Version):

    • check_compatibility/1 - Validate server version from meta response
    • supports_grpc?/1 - Check if version supports gRPC (1.23.0+)
    • grpc_minimum_version/0 - Get minimum gRPC version
  • OIDC Scope Parsing (WeaviateEx.Auth.OIDC):

    • parse_scopes/1 - Parse scope strings (space or comma separated)
  • Microsoft Password Flow Validation (WeaviateEx.Auth.Azure):

    • validate_password_flow/1 - Validate ROPC flow requirements
    • password_flow_configured?/1 - Check if password flow is enabled

Schema/Collections Updates

  • Multi-Tenancy Configuration (WeaviateEx.Schema.MultiTenancyConfig):

    • auto_tenant_creation - Automatically create tenants on first insert
    • auto_tenant_activation - Automatically activate tenants when accessed
    • enabled/0, disabled/0 - Convenience constructors
    • with_auto_creation/0, with_auto_activation/0, fully_automatic/0 - Preset configurations
  • Custom Vectorizer Support (WeaviateEx.API.VectorConfig):

    • custom/2 - Configure custom/unlisted vectorizer modules
    • with_custom_vectorizer/3 - Add custom vectorizer to collection config

Search/Query Enhancements

  • Aggregate near_object (WeaviateEx.API.Aggregate):

    • with_near_object/4 - Aggregate with near_object similarity constraint
  • Aggregate hybrid (WeaviateEx.API.Aggregate):

    • with_hybrid/4 - Aggregate with hybrid search constraint
    • :alpha option for vector/keyword balance
    • :fusion_type option for fusion strategy (:ranked or :relative_score)

Data Types/Objects Improvements

  • Multi-Target References (WeaviateEx.Types.Reference):

    • to/3 - Create reference with optional target vectors
    • multi_target/3 - Create multi-target reference for named vectors
    • from_beacon/1, to_map/1, from_map/1 - Serialization
    • extract_id/1, extract_collection/1 - Extraction utilities
  • Property Validation (WeaviateEx.Validation.Property):

    • validate/2 - Validate property value against data type
    • validate_object/2 - Validate entire object against schema
    • Supports all Weaviate data types including arrays
  • Multi-Dimensional Vectors (WeaviateEx.Types.Vector):

    • validate/1 - Validate 1D or 2D vectors (ColBERT support)
    • shape/1 - Get vector dimensions
    • one_dimensional?/1, multi_dimensional?/1 - Type checks
    • dimensionality/1 - Get inner dimension
    • normalize/1, dot_product/2, cosine_similarity/2 - Vector operations

HTTP Enhancements

  • Rate Limit Header Tracking (WeaviateEx.Protocol.HTTP.RateLimit):
    • from_headers/1 - Extract rate limit info from response headers
    • should_wait?/1 - Check if client should wait
    • remaining_percent/1 - Get remaining request percentage
    • near_limit?/2 - Check if close to rate limit
    • seconds_until_reset/1 - Get seconds until reset
    • summary/1 - Human-readable status

HTTP Transport-Level Retry

  • HTTP Retry Module (WeaviateEx.Protocol.HTTP.Retry):
    • with_retry/2 - Execute function with automatic retry on transport errors
    • calculate_backoff/1 - Exponential backoff calculation (capped at 32 seconds)
    • retryable_transport_error?/1 - Check if error is retryable transport error
    • Retryable errors: connection refused, connection reset, transport timeout, closed, nxdomain
    • Configurable max_retries (default: 3) and base_delay_ms (default: 500)
    • HTTP client now automatically retries on transient transport failures

Per-Operation Timeouts

  • HTTP Client Timeout Integration:
    • Uses WeaviateEx.Config.Timeout module for operation-specific timeouts
    • GET/HEAD requests use query timeout (30s default)
    • POST/PUT/PATCH/DELETE use insert timeout (90s default)
    • GraphQL queries (POST to /v1/graphql) use query timeout
    • Added pool_timeout (5s default) for connection pool
    • Added timeout_config field to WeaviateEx.Client.Config

Batch Memory Safety

  • MAX_STORED_RESULTS Limit (WeaviateEx.Batch.ErrorTracking.Results):
    • Added 100,000 result limit to prevent memory exhaustion
    • Rolling window eviction removes oldest entries when limit reached
    • max_stored_results/0 - Get the configured limit
    • results_within_limit?/1 - Check if results are within limit

Property Value Serialization

  • Payload Serialization (WeaviateEx.Objects.Payload):
    • DateTime values automatically serialized to RFC3339 format (ISO8601)
    • NaiveDateTime values serialized to RFC3339 format (without timezone)
    • GeoCoordinate structs serialized to %{"latitude" => lat, "longitude" => lon}
    • PhoneNumber structs serialized to %{"input" => number, "defaultCountry" => country}
    • Recursive serialization for nested objects and arrays
    • Structs preserved through key normalization for proper serialization

UUID Utilities

  • Beacon URL Extraction (WeaviateEx.Types.UUID):
    • extract_from_beacon/1 - Extract UUID from Weaviate beacon URLs
    • Supports simple format: weaviate://localhost/<uuid>
    • Supports collection-prefixed format: weaviate://localhost/<collection>/<uuid>
    • Automatic UUID validation and lowercase normalization

Changed

  • Enhanced tenant activity status to include all types (active, inactive, hot, cold, frozen, offloaded)
  • Backup list now supports optional sorting parameter

[0.7.1] - 2025-12-29

Added

Custom Headers Support

  • Additional Headers Configuration (WeaviateEx.Client.Config):
    • additional_headers option for custom HTTP headers (e.g., X-OpenAI-Api-Key, X-Cohere-Api-Key)
    • Applied to all HTTP requests via Finch
    • Applied as gRPC metadata with lowercased keys
    • Validation prevents nil header values

Named Vectors in Data Operations

  • Named Vectors Support (WeaviateEx.Objects.Payload):
    • vectors parameter for multiple named vectors (e.g., %{"title_vector" => [0.1, 0.2]})
    • Mutual exclusivity validation between vector (single) and vectors (named)
    • Supported in insert/4, update/4, and replace/4 operations

References During Insert

  • Inline References (WeaviateEx.Objects.Payload):
    • references parameter in Data.insert/4 for inline reference creation
    • Single UUID: %{"hasAuthor" => "uuid-123"}
    • Multiple UUIDs: %{"hasAuthors" => ["uuid-1", "uuid-2"]}
    • Multi-target: %{"relatedTo" => %{target_collection: "Category", uuids: "cat-uuid"}}
    • Automatic beacon format conversion and properties merge

gRPC Exponential Backoff Retry

  • Retry Module (WeaviateEx.GRPC.Retry):
    • with_retry/2 - Execute function with automatic retry on transient errors
    • calculate_backoff/1 - Calculate exponential backoff (capped at 32 seconds)
    • retryable?/1 - Check if error is retryable
    • retryable_status?/1 - Check if gRPC status code is retryable
    • Retryable status codes: UNAVAILABLE (14), RESOURCE_EXHAUSTED (8), ABORTED (10), DEADLINE_EXCEEDED (4)
    • Configurable max_retries (default: 4) and base_delay_ms (default: 1000)

Changed

[0.7.0] - 2025-12-29

Added

Multimodal Search Support

  • NearImage Search (WeaviateEx.Query.NearImage) - Image-based vector search for multimodal collections:
    • new/1 - Create near_image search with base64 data or file path
    • Supports multi2vec-clip, multi2vec-bind, and other image vectorizers
    • Options: :image, :image_file, :certainty, :distance, :target_vectors
    • encode_image_file/1 - Base64 encode image files
    • to_grpc/1, to_graphql/1 - Protocol conversion utilities
  • NearMedia Search (WeaviateEx.Query.NearMedia) - Media-based vector search for multimodal collections:
    • new/2 - Create near_media search with type and media data
    • Supports 5 media types: :audio, :video, :thermal, :depth, :imu
    • Options: :media, :media_file, :certainty, :distance, :target_vectors
    • media_types/0 - List all supported media types
    • to_grpc/1, to_graphql/1 - Protocol conversion utilities
  • Query Builder Integration:
    • Query.near_image/2 - Add image-based search to query
    • Query.near_media/3 - Add media-based search to query
    • Full GraphQL and gRPC support for multimodal queries

gRPC Health Checking

  • gRPC Health Ping (WeaviateEx.GRPC.Services.Health):
    • ping/2 - Lightweight connectivity check returning :ok or {:error, reason}
    • Quick connection verification for startup and heartbeat checks
    • Returns {:error, :no_channel} for nil channels

Server Version Detection

  • Version Module (WeaviateEx.Version) - Parse and validate Weaviate server versions:
    • parse/1 - Parse version strings (handles "v" prefix, prereleases, build metadata)
    • meets_minimum?/2 - Check if version meets minimum requirement
    • get_server_version/1 - Extract version from meta endpoint response
    • validate_server/1 - Validate against minimum supported version (1.27.0)
    • minimum_version/0, minimum_version_string/0 - Get minimum version info
    • format_version/1 - Format version tuple to string

Kubernetes Health Endpoints

  • K8s Probe Support (WeaviateEx.Health):
    • alive?/0, alive?/1 - Liveness probe via /.well-known/live endpoint
    • ready?/0, ready?/1 - Readiness probe via /.well-known/ready endpoint
    • Client and no-client variants for flexibility
    • Returns {:ok, true} or {:ok, false} for consistent handling

Background Batch Processing (Phase 2)

  • Background Batcher (WeaviateEx.Batch.Background) - Process-based async batching:
    • start_link/1 - Start background batcher with configuration
    • add_object/3 - Queue object for background processing
    • add_reference/5 - Queue reference with UUID tracking
    • flush/1 - Trigger immediate flush
    • get_results/1 - Get accumulated results
    • stop/2 - Stop with optional final flush
    • Automatic flushing based on batch size or time interval
    • Concurrent request management with configurable limits
    • UUID tracking for reference ordering
    • Error tracking and callback support
  • Batch Module Integration:
    • Batch.background/3 - Convenience function to start background batcher

Named Vector Query Integration (Phase 2)

  • TargetVectors Enhancements (WeaviateEx.Query.TargetVectors):
    • combine/2 - Create combined target vectors with method (:sum, :average, :minimum)
    • weighted/1 - Create manually weighted target vectors
    • normalize/1 - Normalize various input formats to Config struct
    • to_grpc/1 - Convert to gRPC format
    • Config struct for structured target vector configuration
  • Query Builder Integration:
    • Query.near_vector/3 now accepts :target_vectors option
    • Query.near_text/3 now accepts :target_vectors option
    • Query.near_object/3 now accepts :target_vectors option

Advanced Hybrid Search (Phase 2)

  • HybridVector Enhancements (WeaviateEx.Query.HybridVector):
    • near_text/2 - Text-based vector search with Move operations
    • near_vector/2 - Vector-based search with target vectors
    • to_grpc/1 - Convert to gRPC format for query execution
    • Support for :move_to, :move_away_from, :target_vectors options
    • query field for better naming consistency
  • Query.hybrid/3 Enhancements:
    • :vector option accepts HybridVector configuration
    • :properties option for BM25 component
    • :target_vectors option for multi-vector collections

Connection Pool Configuration (Phase 2)

  • Connection Config (WeaviateEx.Config.Connection) - Fine-tune connection settings:
    • new/1 - Create pool config with size, max connections, timeouts
    • to_finch_opts/1 - Convert to Finch HTTP client options
    • to_grpc_opts/1 - Convert to gRPC channel options
    • Options: :pool_size, :max_connections, :pool_timeout, :max_idle_time

Azure OIDC Support (Phase 2)

  • Azure Authentication (WeaviateEx.Auth.Azure) - Azure-specific OIDC handling:
    • azure_endpoint?/1 - Detect Azure/Microsoft token endpoints
    • default_scopes/1 - Generate Azure-style .default scopes
    • apply_azure_defaults/1 - Auto-configure Azure authentication
    • detect_version/1 - Detect Azure v1/v2 endpoint version
    • build_token_params/2 - Build Azure-specific token parameters

Dynamic gRPC Message Size (Phase 2)

  • Version Module (WeaviateEx.Version):
    • get_grpc_max_message_size/1 - Extract gRPC max message size from server meta

Filter Enhancements (Phase 4)

  • Property Length Filtering (WeaviateEx.Filter):

    • by_property_length/3 - Filter by string or array length
    • len/1 - Create length path helper (e.g., len("title"))
    • Operators: :equal, :not_equal, :greater_than, :greater_or_equal, :less_than, :less_or_equal
    • Use for filtering empty strings, minimum content length, array element counts
  • Reference Path Traversal (WeaviateEx.Filter.RefPath):

    • through/2, through/3 - Build multi-level reference paths
    • property/4 - Terminate path with property filter
    • build_path/2 - Build path list from segments
    • to_path/1 - Get path without final property
    • depth/1 - Get number of hops in reference path
    • Enables deep filtering: RefPath.through("hasAuthor", "Author") |> RefPath.through("worksAt", "Company") |> RefPath.property("industry", :equal, "Tech")
  • Multi-Target Reference Filters (WeaviateEx.Filter.MultiTargetRef):

    • new/2 - Create multi-target reference filter builder
    • where/4 - Add property filter on specific target collection
    • deep_where/2 - Deep path filtering with RefPath
    • as_ref_path/1 - Convert to RefPath for chaining
    • Filter different target collections: MultiTargetRef.new("relatedTo", "Article") |> MultiTargetRef.where("title", :like, "Tech*")
  • Filter Module Integration:

    • by_ref_path/4 - Create filter from RefPath
    • by_ref_multi_target/5 - Create multi-target reference filter

Query Reference Enhancements (Phase 4)

  • Multi-Target References (WeaviateEx.Query.QueryReference):

    • multi_target/3 - Create query for specific target collection
    • multi_target?/1 - Check if reference is multi-target
    • Query different targets: QueryReference.multi_target("relatedTo", "Article", return_properties: ["title"])
  • Return Metadata Option:

    • return_metadata option for references
    • Accepts :full, :common, or list of atoms (:uuid, :distance, :certainty, :creation_time, etc.)
    • Include metadata in referenced objects: QueryReference.new("hasAuthor", return_metadata: [:uuid, :distance])

Named Vector Config Updates (Phase 4)

  • Update Configuration (WeaviateEx.API.NamedVectors):
    • update_config/2 - Create update config for existing named vector
    • update_to_api/1 - Convert update to API format
    • build_update_config/1 - Build combined update for multiple vectors
    • Vector index options: :ef, :dynamic_ef_min, :dynamic_ef_max, :dynamic_ef_factor, :flat_search_cutoff
    • Quantizer options: :type (:pq, :bq, :sq), :segments, :centroids, :training_limit, :rescore_limit
    • Example: NamedVectors.update_config("title_vector", vector_index: [ef: 200], quantizer: [type: :pq, segments: 128])

Custom Provider Configurations (Phase 4)

  • Custom Generative Providers (WeaviateEx.API.GenerativeConfig):

    • custom/2 - Create config for unlisted generative AI providers
    • Automatic snake_case to camelCase conversion
    • Pass any options: api_endpoint, model, api_key_header, temperature, max_tokens, etc.
    • Example: GenerativeConfig.custom("my-llm", api_endpoint: "https://llm.example.com", model: "custom-gpt")
  • Reranker Configuration Module (WeaviateEx.API.RerankerConfig):

    • cohere/2 - Cohere Rerank configuration
    • transformers/1 - Local transformers reranker
    • voyageai/2 - Voyage AI reranker
    • jinaai/2 - Jina AI reranker
    • custom/2 - Custom/unlisted reranker providers
    • none/0 - Disable reranking
    • All providers support :base_url option for custom endpoints

Changed

  • Updated Query struct to include near_image and near_media fields
  • Enhanced GraphQL query building with nearImage and nearMedia support

Test Infrastructure

Supertester Integration

  • Added supertester ~> 0.4.0 as test dependency for robust, async-safe testing
  • Eliminated all Process.sleep calls from unit tests using supertester-style patterns:
    • token_manager_test.exs: Replaced 8 sleep calls with deterministic state polling
    • dynamic_test.exs: Replaced 1 sleep call with state-based synchronization
    • state_test.exs: Replaced 1 sleep call with explicit time-wait pattern
  • New Test Helpers (WeaviateEx.TestHelpers):
    • wait_until/2 - Polls until condition is met with configurable timeout/interval
    • wait_for_genserver_state/3 - Waits for GenServer state to match a condition
    • trigger_and_wait/4 - Sends message and waits for state change
  • All 2151 tests pass with full async execution support

Stats

  • Phase 1-4: 20+ new feature modules/functions
  • Phase 4: 6 new modules (RefPath, MultiTargetRef, RerankerConfig) + major enhancements
  • Full test coverage for all new features
  • Backward compatible with v0.6.0
  • Test suite now fully async-safe with zero timing-based synchronization

[0.6.0] - 2025-12-29

Added

Quantizer Support

  • Quantizer Module (WeaviateEx.API.Quantizer) - Typed structs for vector quantization:
    • PQConfig - Product Quantization with segments, centroids, encoder configuration
    • BQConfig - Binary Quantization with cache and rescore options
    • SQConfig - Scalar Quantization with training limit configuration
    • RQConfig - Rotational Quantization with configurable bit depth
    • Each config provides new/1, to_api/1, from_api/1 for serialization round-trips
    • Convenience aliases: Quantizer.pq/1, Quantizer.bq/1, Quantizer.sq/1, Quantizer.rq/1
    • detect_type/1 - Auto-detect quantizer type from API response
    • from_api/1 - Parse any quantizer from API response

Core Vectorizer Modules

  • Text2VecAWS (WeaviateEx.API.Vectorizers.Text2VecAWS) - AWS vectorizer support:
    • Bedrock service with model selection (Titan, Cohere, etc.)
    • SageMaker service with endpoint, target model, and variant options
    • new/1, to_api/1, from_api/1, vectorizer_name/0
  • Text2VecGoogle (WeaviateEx.API.Vectorizers.Text2VecGoogle) - Google vectorizer support:
    • Vertex AI service with project ID and region
    • Gemini service (Google AI Studio) with auto-configured endpoint
    • Task type, title property, and dimensions configuration
    • new/1, to_api/1, from_api/1, vectorizer_name/0
  • Text2VecWeaviate (WeaviateEx.API.Vectorizers.Text2VecWeaviate) - Weaviate-hosted embeddings:
    • Model selection for Weaviate-managed embedding service
    • Base URL configuration for custom deployments
    • new/1, to_api/1, from_api/1, vectorizer_name/0
  • Img2VecNeural (WeaviateEx.API.Vectorizers.Img2VecNeural) - Neural image vectorizer:
    • Image fields configuration for multi-image objects
    • new/1, to_api/1, from_api/1, vectorizer_name/0

OIDC Token Management

  • OIDC Module (WeaviateEx.Auth.OIDC) - OpenID Connect support:
    • discover/1 - OIDC configuration discovery from issuer URL
    • get_token/2 - Token exchange for client_credentials and password grants
    • refresh_token/2 - Token refresh using refresh tokens
    • OIDC.Config struct for provider configuration
    • OIDC.TokenResponse struct with expiration helpers:
      • expires_at/1 - Calculate expiration timestamp
      • expired?/1 - Check if token is expired
      • expiring_soon?/2 - Check if token expires within buffer time
  • TokenManager GenServer (WeaviateEx.Auth.TokenManager) - Automatic token management:
    • start_link/1 - Start with OIDC config or issuer URL
    • get_token/1 - Get current valid token
    • get_access_token/1 - Get just the access token string
    • force_refresh/1 - Force immediate token refresh
    • Automatic background token refresh before expiration
    • Configurable refresh buffer (default: 60 seconds before expiry)
    • child_spec/1 - Easy integration with supervision trees
  • Auth Module Extensions:
    • get_oidc_headers/1 - Get authorization headers from TokenManager
    • oidc_config/1 - Create OIDC configuration for TokenManager

Generative Search Integration

  • Query.Generate Module (WeaviateEx.Query.Generate) - Combine search with AI generation:
    • new/1 - Create generative query for collection
    • near_text/3, near_vector/3, near_object/3 - Vector similarity with generation
    • bm25/3, hybrid/3 - Keyword and hybrid search with generation
    • single/2 - Single-object generation (generate per result)
    • grouped/2 - Grouped generation (generate once for all results)
    • return_properties/2, where/2, limit/2, offset/2, tenant/2
    • additional/2 - Request additional metadata fields
    • to_graphql/1 - Convert to GraphQL query string
    • execute/2 - Execute query and parse results
    • parse_response/2 - Parse API response to GenerativeResult
  • GenerativeResult Struct (WeaviateEx.Query.GenerativeResult):
    • objects - List of result objects with properties
    • generated - Grouped generation result
    • generated_per_object - List of per-object generation results
  • Query Module Integration:
    • Query.generate/4 - Convert Query struct to Generate query

Nested Properties Support

  • Property.Nested Module (WeaviateEx.Property.Nested) - Nested object schema support:
    • new/1 - Create nested property with name, data_type, nested_properties
    • to_api/1 - Serialize to Weaviate API format
    • from_api/1 - Parse from API response (supports recursive nesting)
    • valid?/1 - Validate nested property configuration
    • object_type?/1 - Check if property is object or object_array type
  • Property Module Integration:
    • Property.object/3 - Create object property with nested properties
    • Property.object_array/3 - Create object array property with nested properties
  • DataType Module:
    • :object and :object_array data types for nested structures

Batch Improvements

  • Concurrent Batch Operations (WeaviateEx.Batch.Concurrent):
    • insert_many/4 - Parallel batch insertion using Task.async_stream
    • Configurable: max_concurrency, batch_size, ordered, timeout
    • split_into_batches/2 - Split objects into batches
    • aggregate_results/2 - Combine results from parallel batches
    • retryable_error?/1 - Detect retryable errors
    • Result struct with all_successful?/1, has_failures?/1, summary/1
  • Batch Queue (WeaviateEx.Batch.Queue) - FIFO queue for failure tracking:
    • new/0 - Create empty queue
    • enqueue/2 - Add object to pending queue
    • dequeue_batch/2 - Get batch of objects for processing
    • mark_failed/3 - Move object to failed queue with reason
    • requeue_failed/2 - Move failed objects back to pending (respects max_retries)
    • pending_count/1, failed_count/1, empty?/1
    • FailedObject struct with retry count and failure timestamp
  • Rate Limit Detection (WeaviateEx.Batch.RateLimit):
    • detect/1 - Detect rate limiting from HTTP response
    • rate_limited?/1 - Check if error indicates rate limiting
    • calculate_backoff/1 - Calculate exponential backoff with jitter
    • extract_retry_after/1 - Extract retry-after from headers
    • Provider-specific patterns: OpenAI (429), Cohere (rate limit errors)
  • Server Queue Monitoring (WeaviateEx.API.Cluster):
    • batch_stats/1 - Get cluster batch statistics (queue_length, rate_per_second, failed_count)
    • Aggregates stats from all nodes for dynamic batch sizing
  • Dynamic Batch Sizing (WeaviateEx.Batch.Dynamic):
    • :monitor_server_stats option - Enable server queue monitoring
    • :poll_interval option - Configure stats polling frequency
    • get_server_batch_stats/1 - Get current server stats from GenServer

gRPC Batch Streaming (Weaviate 1.34+)

  • Batch Stream Service (WeaviateEx.GRPC.Services.BatchStream) - Low-level gRPC streaming:
    • open/2 - Open bidirectional gRPC stream
    • send_objects/2 - Send batch objects over stream
    • send_references/2 - Send cross-references over stream
    • receive_results/2 - Receive batch results with timeout
    • close/1 - Close stream and clean up resources
    • start_message/1, stop_message/0, data_message/2 - Protocol message builders
  • High-Level Batch Stream (WeaviateEx.Batch.Stream) - Client-side buffered streaming:
    • new/3 - Create stream session with buffer configuration
    • add/2 - Add object to buffer (auto-flush when full)
    • flush/1 - Manually flush buffer to server
    • close/1 - Close stream and return all results
    • Configurable: buffer_size, flush_interval_ms, auto_flush, auto_reconnect
    • Automatic connection management and error recovery

RBAC Enhancements

  • Scope Permissions (WeaviateEx.API.RBAC.Scope) - Fine-grained access control:
    • all_collections/0 - Create scope for all collections
    • collection/1 - Create scope for specific collection
    • collections/1 - Create scope for list of collections
    • with_tenants/2 - Add tenant restrictions to scope
    • with_shards/2 - Add shard restrictions to scope
    • to_api/1, from_api/1 - API serialization
  • Permission Builder (WeaviateEx.API.RBAC.Permission) - Permission construction:
    • new/3 - Create permission with resource, action, and scope
    • read_collection/1, manage_data/1 - Common permission shortcuts
    • admin/0 - Full administrative permissions
    • viewer/0 - Read-only permissions
    • to_api/1, from_api/1 - API serialization
  • Database User Management (WeaviateEx.API.Users.DB) - DB-backed users:
    • create/3 - Create DB user (returns API key)
    • get/3 - Get DB user details
    • list/2 - List all DB users
    • delete/3 - Delete DB user
    • rotate_api_key/3 - Rotate user's API key
    • assign_roles/4, revoke_roles/4 - Role management
  • OIDC User Management (WeaviateEx.API.Users.OIDC) - OIDC-backed users:
    • get/3 - Get OIDC user details
    • list/2 - List OIDC users
    • assign_roles/4, revoke_roles/4 - Role management
    • Note: OIDC users cannot be created/deleted via API

Additional Vectorizers

Debug Module

  • Debug API (WeaviateEx.Debug) - Protocol comparison and troubleshooting:
    • get_object_rest/4 - Fetch object via REST/HTTP protocol
    • get_object_grpc/4 - Fetch object via gRPC protocol
    • compare_protocols/4 - Compare REST and gRPC responses with detailed diff
    • connection_info/1 - Get HTTP and gRPC connection diagnostics
  • ObjectCompare (WeaviateEx.Debug.ObjectCompare) - Deep object comparison:
    • compare/2 - Compare two objects and detect differences
    • diff/2 - Generate list of differences between objects
    • format_diff/1 - Format differences as human-readable report
  • RequestLogger (WeaviateEx.Debug.RequestLogger) - Request logging GenServer:
    • start_link/1 - Start logger with optional name
    • enable/1, disable/1 - Toggle request logging
    • log_request/2 - Log HTTP/gRPC request details
    • get_logs/2 - Retrieve logs with optional filters (protocol, min_duration_ms)
    • export_logs/3 - Export logs to file (JSON or text format)
    • clear_logs/1 - Clear all logged requests
  • Main Module Integration:

Backup Enhancements

  • RBAC Restore Options (WeaviateEx.API.Backup):
    • roles_restore option - Restore role definitions from backup
    • users_restore option - Restore user role assignments from backup
    • overwrite_alias option - Overwrite existing collection aliases during restore
  • Location Struct Support (WeaviateEx.API.Backup):
    • create/4 now accepts Location structs (Filesystem, S3, GCS, Azure) directly
    • restore/4 now accepts Location structs directly
    • Dynamic location configuration for programmatic backend selection
  • Backup Config Enhancement (WeaviateEx.Backup.Config.Create):
    • Added chunk_size option - Configure backup chunk size in megabytes

Connection Management

  • Pool Configuration (WeaviateEx.Client.Pool) - Connection pool settings:
    • new/1 - Create pool config with size, overflow, strategy, timeouts
    • default/0 - Default pool configuration
    • default_http/0 - HTTP-optimized pool (larger pool for parallel requests)
    • default_grpc/0 - gRPC-optimized pool (smaller pool due to multiplexing)
    • to_finch_opts/1 - Convert to Finch HTTP client options
    • to_grpc_opts/1 - Convert to gRPC channel options
    • Configurable: size, overflow, strategy (:fifo/:lifo), timeout, idle_timeout, max_age
  • Client State Tracking (WeaviateEx.Client.State) - Lifecycle state management:
    • new/0 - Create initial state with timestamps
    • connected/1, disconnected/2, closed/1 - State transitions
    • record_request/1 - Track successful requests
    • record_error/2 - Track errors with details
    • Status tracking: :initializing, :connected, :disconnected, :closed
    • Statistics: request_count, error_count, created_at, last_used_at, last_error
  • Client Lifecycle (WeaviateEx.Client):
    • close/1 - Close client and release all connections
    • closed?/1 - Check if client has been closed
    • status/1 - Get current client status
    • stats/1 - Get client statistics (request/error counts, timestamps)
    • with_client/2 - Execute function with auto-managed client lifecycle
    • Added state field to client struct for tracking lifecycle
  • ClosedClientError (WeaviateEx.Error.ClosedClientError) - Exception for closed client:
    • Raised when operations are attempted on a closed client
    • Includes closed_at timestamp for debugging

[0.5.0] - 2025-12-28

Added

Proxy Support

  • Proxy Configuration (WeaviateEx.Config.Proxy) - HTTP, HTTPS, and gRPC proxy support:
    • new/1 - Create proxy config with explicit options
    • from_env/0 - Read from environment variables (HTTP_PROXY, HTTPS_PROXY, GRPC_PROXY)
    • configured?/1 - Check if any proxy is configured
    • http_proxy_for/2 - Get appropriate proxy for a URL scheme
    • to_finch_opts/1 - Convert to Finch HTTP client options
    • to_grpc_opts/1 - Convert to gRPC channel options
  • Environment variable reading with case-insensitive support (uppercase takes precedence)
  • Automatic fallback from HTTPS to HTTP proxy when HTTPS not configured

Inverted Index Configuration

  • Inverted Index Config (WeaviateEx.API.InvertedIndexConfig) - Collection index configuration:
    • bm25/1 - BM25 algorithm configuration with b and k1 parameters
    • stopwords/1 - Stopwords configuration with preset, additions, and removals
    • index_timestamps/1 - Enable/disable timestamp indexing
    • index_property_length/1 - Enable/disable property length indexing
    • index_null_state/1 - Enable/disable null state indexing
    • cleanup_interval_seconds/1 - Set cleanup interval
    • build/1 - Build complete inverted index configuration
    • merge/2 - Merge two configurations
    • validate/1 - Validate configuration values
  • Supported stopwords presets: en, none
  • BM25 parameter validation (b: 0-1, k1: positive)

Generative Search Configuration

  • Generative Config (WeaviateEx.API.GenerativeConfig) - 13+ provider configurations:
    • openai/1 - OpenAI GPT models
    • azure_openai/1 - Azure OpenAI deployments
    • cohere/1 - Cohere Command models
    • anthropic/1 - Anthropic Claude models
    • mistral/1 - Mistral AI models
    • google/1 - Google Gemini/PaLM models
    • aws/1 - AWS Bedrock/SageMaker models
    • ollama/1 - Ollama local models
    • databricks/1 - Databricks endpoints
    • nvidia/1 - NVIDIA NIM models
    • friendliai/1 - FriendliAI models
    • xai/1 - XAI Grok models
    • anyscale/1 - Anyscale models
    • contextualai/1 - ContextualAI models
  • providers/0 - List all supported providers
  • provider_module/1 - Get Weaviate module name for provider
  • Full parameter support: model, temperature, maxTokens, baseURL, topK, topP

Collection Aliases API

  • Aliases API (WeaviateEx.API.Aliases) - Zero-downtime collection management (requires Weaviate v1.32.0+):
    • create/3 - Create an alias for a collection
    • delete/2 - Delete an alias
    • update/3 - Update alias to point to different collection
    • get/2 - Get alias details
    • list/2 - List all aliases (optionally filter by collection)
    • exists?/2 - Check if alias exists
    • minimum_version/0 - Get minimum required Weaviate version
  • Alias Struct (WeaviateEx.API.Aliases.Alias):
    • from_api/1 - Parse API response to struct
    • Fields: alias, collection

ZSTD Compression Options

  • Extended Compression (WeaviateEx.Backup.Compression) - Additional compression algorithms:
    • :zstd_default - Balanced ZSTD compression
    • :zstd_best_speed - Fast ZSTD compression
    • :zstd_best_compression - Maximum ZSTD compression
    • :no_compression - Disable compression
    • gzip?/1 - Check if compression level uses GZIP
    • zstd?/1 - Check if compression level uses ZSTD
  • ZSTD provides faster compression with better ratios than GZIP

Query Move Integration

  • Move in near_text (WeaviateEx.Query) - Semantic direction control:
    • near_text/3 now supports :move_to and :move_away options
    • Move concepts toward or away from specific terms
    • Accepts WeaviateEx.Query.Move structs or keyword options
    • Proper GraphQL generation for move parameters
  • Example: Query.near_text("technology", move_to: [concepts: ["AI"], force: 0.8])

Changed

  • Updated test count to 1575 tests
  • Enhanced Query module with Move struct integration
  • Extended Backup.Compression with ZSTD support while maintaining backward compatibility

Stats

  • 6 new feature modules
  • Full test coverage for all new features
  • Backward compatible with v0.4.0

[0.4.0] - 2025-12-28

Added

Cluster Management

  • Cluster API (WeaviateEx.API.Cluster) - Complete cluster management:
    • nodes/2 - Get cluster node information with optional verbose output
    • shards/2 - Get shards for a collection
    • statistics/1 - Get cluster-wide statistics
    • replicate/4 - Trigger shard replication between nodes
    • list_replications/2 - List ongoing replication operations
    • get_replication/3 - Get replication operation status
    • cancel_replication/2 - Cancel an ongoing replication
    • delete_replication/2 - Delete a replication operation record
    • wait_for_replications/2 - Wait for all replications to complete
  • Cluster Type Structs:

Query Rerank & GroupBy Integration

  • Query.Rerank enhancements:
    • to_map/1 - Convert to map format for gRPC
    • valid?/1 - Validate rerank configuration
    • Improved to_graphql/1 with proper string escaping
  • Query.GroupBy enhancements:
    • Support for list paths (nested properties)
    • to_map/1 - Convert to map format for gRPC
    • valid?/1 - Validate group_by configuration
  • Query Builder Integration:
    • Query.rerank/2 - Add rerank to query
    • Query.group_by/2 - Add group_by to query
    • Works with all query types: near_text, near_vector, hybrid, bm25

Batch Advanced Features

  • wait_for_vector_indexing (WeaviateEx.Batch):
    • Poll until all vectors are indexed
    • Configurable timeout, poll interval, max failures
    • Optional shard filtering
  • Auto UUID Generation (WeaviateEx.Batch.FixedSize):
  • Multi-Target References (WeaviateEx.Batch.FixedSize):
    • add_reference/6 now supports list of targets
    • Each target specifies collection and uuid
  • DeleteResult Struct (WeaviateEx.Batch.DeleteResult):
    • Typed result for batch delete operations
    • from_api/1 - Parse API response
    • all_successful?/1, has_failures?/1
    • failed_objects/1, successful_objects/1
    • summary/1 - Human-readable summary
    • DeletedObject submodule for individual objects

Reranker Configurations

  • 6 Reranker Types in WeaviateEx.API.VectorConfig:
    • reranker_cohere/1 - Cohere reranker (existing)
    • reranker_transformers/1 - Local transformers reranker
    • reranker_voyageai/1 - VoyageAI reranker
    • reranker_jinaai/1 - Jina AI reranker
    • reranker_nvidia/1 - NVIDIA reranker
    • reranker_contextualai/1 - Contextual AI reranker
  • Builder Function: with_reranker/3 - Add reranker to collection config

Backup & Restore Module

  • Backup API (WeaviateEx.API.Backup) - Complete backup and restore functionality:
    • create/4 - Create backup with include/exclude collections, config, wait_for_completion
    • restore/4 - Restore backup with include/exclude collections, config, wait_for_completion
    • get_create_status/3 - Get backup creation status
    • get_restore_status/3 - Get restore operation status
    • list/2 - List all backups for a storage backend
    • cancel/3 - Cancel in-progress backup
    • wait_for_completion/5 - Poll until operation completes with configurable timeout
  • Storage Backend Enum (WeaviateEx.Backup.Storage) - 4 supported backends:
    • :filesystem - Local filesystem storage
    • :s3 - Amazon S3 or S3-compatible storage
    • :gcs - Google Cloud Storage
    • :azure - Azure Blob Storage
    • Helper functions: all/0, valid?/1, to_api_path/1, from_api/1
  • Compression Level Enum (WeaviateEx.Backup.Compression):
    • :default - Balanced compression
    • :best_speed - Faster compression, larger files
    • :best_compression - Slower compression, smaller files
  • Backup Status Types (WeaviateEx.Backup.Status):
    • Status values: :started, :transferring, :transferred, :success, :failed, :canceled
    • Response structs: CreateResponse, RestoreResponse, BackupInfo
    • Helper functions: completed?/1, success?/1, in_progress?/1
  • Backup Configuration (WeaviateEx.Backup.Config):
    • Create struct with cpu_percentage and compression options
    • Restore struct with cpu_percentage option
    • Factory functions: create/1, restore/1
  • Storage Location Structs (WeaviateEx.Backup.Location):
    • Filesystem - Local path configuration
    • S3 - Bucket, path, endpoint, region, credentials, SSL
    • GCS - Bucket, path, project ID, credentials
    • Azure - Container, path, connection string
    • Factory functions: filesystem/1, s3/3, gcs/3, azure/3
  • Backup Error Types (WeaviateEx.Error):
    • backup_not_found/2, backup_already_exists/2, backup_failed/2
    • restore_failed/2, backup_timeout/2, invalid_backend/1
  • Main Module Backup Functions:
    • create_backup/4, restore_backup/4, list_backups/2
    • cancel_backup/3, get_backup_status/3, get_restore_status/3

Role-Based Access Control (RBAC)

  • RBAC Module (WeaviateEx.RBAC) - Complete role-based access control support:
    • Actions - Action type definitions and conversions for all 11 permission types
    • Permission - Permission struct with encoding/decoding for API communication
    • Permissions - Builder API for constructing permissions fluently
    • Role - Role struct with permission management
  • RBAC API (WeaviateEx.API.RBAC) - Role CRUD operations:
    • list_roles/1 - List all roles
    • get_role/2 - Get role by name
    • create_role/3 - Create role with permissions
    • delete_role/2 - Delete a role
    • add_permissions/3 - Add permissions to role
    • remove_permissions/3 - Remove permissions from role
    • has_permissions?/3 - Check if role has permissions
    • get_users_for_role/2 - Get users assigned to role
    • get_groups_for_role/2 - Get groups assigned to role
    • exists?/2 - Check if role exists
  • 11 Permission Types: collections, data, tenants, roles, users, groups, cluster, nodes, backups, replicate, alias
  • Type-safe Permissions Builder (WeaviateEx.RBAC.Permissions):
    • collections/2 - Collection schema permissions
    • data/3 - Data CRUD permissions with tenant/object filters
    • tenants/3 - Tenant management permissions
    • roles/2 - Role management permissions
    • users/2 - User management permissions
    • groups/2 - OIDC group permissions
    • cluster/1 - Cluster info permissions
    • nodes/1 - Node info permissions (minimal/verbose)
    • backups/1 - Backup management permissions
    • replicate/2 - Replication permissions
    • alias_permission/2 - Collection alias permissions

User Management

  • User Structs (WeaviateEx.Users.User):
    • User.DB - Database-managed users with API key
    • User.OIDC - OIDC-managed users with groups
    • User.Own - Current authenticated user info
  • Users API (WeaviateEx.API.Users) - User lifecycle management:
    • create/2 - Create DB user (returns API key)
    • get/2 - Get user by ID
    • list_all/1 - List all users
    • delete/2 - Delete user
    • activate/2 - Activate user
    • deactivate/2 - Deactivate user
    • rotate_key/2 - Rotate API key
    • assign_roles/3 - Assign roles to user
    • revoke_roles/3 - Revoke roles from user
    • get_assigned_roles/2 - Get user's roles
    • get_my_user/1 - Get current user info

Group Management

  • Group Struct (WeaviateEx.Groups.Group) - OIDC group representation
  • Groups API (WeaviateEx.API.Groups) - OIDC group operations:
    • list_known/1 - List known OIDC groups
    • get_assigned_roles/2 - Get roles assigned to group
    • assign_roles/3 - Assign roles to group
    • revoke_roles/3 - Revoke roles from group

Error Types

  • Cluster Errors (WeaviateEx.Error):
    • node_not_found/1 - Node not found in cluster
    • shard_not_found/2 - Shard not found in collection
    • replication_failed/2 - Replication operation failed
    • replication_timeout/1 - Replication timed out
    • cluster_not_ready/0 - Cluster not ready
    • vector_indexing_timeout/1 - Vector indexing timed out
  • RBAC-specific Errors (WeaviateEx.Error):
    • rbac_error/3 - Create RBAC error with category
    • role_not_found/1 - Role not found error
    • permission_denied/2 - Permission denied error
    • user_not_found/1 - User not found error
    • invalid_permission/1 - Invalid permission error

Main Module Delegations

  • Cluster convenience functions: cluster_nodes/2, cluster_shards/2, cluster_statistics/1
  • Replication functions: replicate_shard/4, list_replications/2, get_replication/3
  • Replication control: cancel_replication/2, delete_replication/2, wait_for_replications/2
  • list_roles/1, get_role/2, create_role/3, delete_role/2
  • create_user/2, get_user/2, list_users/1, delete_user/2, get_my_user/1
  • list_groups/1, assign_group_roles/3, revoke_group_roles/3

gRPC Protocol Support

  • gRPC Channel Management (WeaviateEx.GRPC.Channel) - Persistent connection management:
    • connect/3 - Establish gRPC channel with TLS support
    • disconnect/1 - Clean channel shutdown
    • build_metadata/1 - Auth metadata for gRPC calls
    • Automatic reconnection handling
  • gRPC Services - High-performance data operations:
  • Protocol Buffer Definitions - Generated from Weaviate v1 protos:
    • 11 proto files in priv/protos/v1/
    • Generated Elixir modules in lib/weaviate_ex/grpc/generated/v1/
  • gRPC Error Handling (WeaviateEx.Error):
    • from_grpc_status/3 - Map gRPC status codes to error types
    • from_grpc_error/1 - Convert gRPC errors to WeaviateEx.Error
    • grpc_retryable?/1 - Identify retryable gRPC errors
  • gRPC Retry Logic (WeaviateEx.Retry):
    • Retry on UNAVAILABLE, RESOURCE_EXHAUSTED, ABORTED status codes
    • Exponential backoff with jitter
  • Client Configuration (WeaviateEx.Client.Config):
    • grpc_host - gRPC endpoint hostname
    • grpc_port - gRPC port (default: 50051)
    • grpc_max_message_size - Maximum message size for gRPC calls
    • use_tls?/1 - TLS detection for gRPC connections

Changed

  • Updated documentation with new module groups
  • Improved test coverage for all new features
  • Hybrid Architecture: gRPC for data operations (queries, batch, aggregations), HTTP retained for schema operations (Weaviate gRPC API doesn't support schema management)
  • Client Connection (WeaviateEx.Client):
    • connect/1 now establishes both HTTP (Finch) and gRPC channels
    • New grpc_channel field in client struct
    • Automatic gRPC fallback to HTTP when channel unavailable
  • Query Execution (WeaviateEx.Query):
    • execute/2 with client uses gRPC when available
    • execute/1 without client uses HTTP (backwards compatible)
  • Batch Operations (WeaviateEx.API.Batch):
    • create_objects/3 uses gRPC when client has gRPC channel
    • delete_objects/2 uses gRPC for batch deletes
  • Aggregations (WeaviateEx.API.Aggregate):
    • Simple count/group_by use gRPC
    • Complex aggregations (multiple properties) use GraphQL
  • Tenants (WeaviateEx.API.Tenants):
    • list/2, get/3, exists?/3 use gRPC
    • Create/update/delete remain HTTP (not in gRPC API)

Dependencies

  • Added {:grpc, "~> 0.9"} - Elixir gRPC client
  • Added {:protobuf, "~> 0.13"} - Protocol Buffer support
  • Retained {:finch, "~> 0.18"} - For schema operations and HTTP fallback

Stats

  • Full gRPC support for data operations
  • Complete Backup & Restore support with 4 storage backends
  • Complete RBAC support matching Python client functionality
  • Complete Cluster management with replication support
  • Backwards compatible - existing code continues to work

[0.3.0] - 2025-12-28

Added

Query Enhancements

Reference Operations

  • References API (WeaviateEx.API.References) - Full cross-reference CRUD:
    • add/6 - Add single reference
    • delete/6 - Delete reference
    • replace/6 - Replace all references on a property
    • add_many/4 - Batch add references
  • ReferenceToMulti (WeaviateEx.Data.ReferenceToMulti) - Multi-target reference type with to_beacons/1

Generative AI Enhancements

  • Typed Provider Configs (WeaviateEx.Generative.Config) - Full configuration structs for 14 providers:
    • OpenAI, Azure OpenAI, Anthropic, Cohere
    • AWS Bedrock/SageMaker, Google Vertex/Gemini
    • Mistral, Ollama, XAI, ContextualAI, Anyscale
    • NEW: NVIDIA NIM, Databricks, FriendliAI
  • Generative Results (WeaviateEx.Generative.Result) - Typed result structures:
    • Single - Single prompt result with metadata and debug
    • Grouped - Grouped task result
    • GenerativeObject - Object with generative result
    • ResponseParser - Parse API responses to typed structs
  • Generative Parameters (WeaviateEx.Generative.Parameters) - Multimodal support:
    • SinglePrompt / GroupedTask with image support
    • image_properties, non_blob_properties options
    • metadata and debug options
  • 20+ AI Providers - Added nvidia, databricks, friendliai to supported providers

Batch Operations

  • Error Tracking (WeaviateEx.Batch.ErrorTracking) - Detailed error tracking:
    • ErrorObject - Failed object details with retry count
    • ErrorReference - Failed reference details
    • Results - Aggregated results with helpers
  • Batch Retry (WeaviateEx.Batch.BatchRetry) - Smart retry logic:
    • Rate limit detection
    • Exponential backoff calculation
    • with_retry/2 wrapper function
  • Fixed Size Batcher (WeaviateEx.Batch.FixedSize) - Fixed-size batch processor

Tenant Extensions

  • freeze/3 - Set tenant to FROZEN state
  • offload/3 - Set tenant to OFFLOADED state

Multi-Vector Support

  • Multi-Vector API (WeaviateEx.API.MultiVector) - ColBERT-style embeddings:
    • muvera_encoding/1 - Muvera encoding configuration
    • multi_vector_config/1 - Multi-vector index configuration
    • self_provided/1 - Self-provided multi-vectors
    • text2colbert_jinaai/1 - Jina ColBERT vectorizer
    • multi2multivec_jinaai/1 - Jina multi-modal vectorizer

Vectorizers

  • New Vectorizers (Dec 2025 Python client sync):
    • text2vec_voyageai/1 - VoyageAI (voyage-3.5, voyage-3-large, voyage-context-3)
    • text2vec_morph/1 - Morph embeddings
    • text2vec_model2vec/1 - Model2Vec embeddings
    • text2vec_aws_bedrock/1 - AWS Bedrock embeddings
    • text2vec_aws_sagemaker/1 - AWS SageMaker endpoints
    • text2vec_google_vertex/1 - Google Vertex AI
    • text2vec_google_gemini/1 - Google AI Studio (Gemini)
    • reranker_cohere/1 - Cohere reranker with baseURL

Other

  • Object TTL Configuration (WeaviateEx.Config.ObjectTTL)
  • OpenAI O1/O3 Support: verbosity and reasoning_effort parameters
  • Cohere Enhancements: dimensions parameter for embeddings
  • CI Infrastructure: Full GitHub Actions workflow

Changed

  • Updated Docker Compose to Weaviate 1.28.14
  • Copied full CI infrastructure from Python client (ci/ directory)
  • Added default version to CI start/stop scripts

Stats

  • 694 tests passing (up from 536)
  • Full Python client feature parity for core operations

[0.2.0] - 2025-10-19

Added

  • Embedded Mode: Download and manage Weaviate embedded binary lifecycle with WeaviateEx.start_embedded/1 and WeaviateEx.stop_embedded/1
  • Comprehensive Docker Environment: Full Docker Compose profiles from Python client (single node, modules, RBAC, async, cluster, proxy, backup, WCS, Okta)
  • Mix Tasks for Docker Management:
  • Batch API Enhancements: Comprehensive batch operations with detailed summaries, error tracking, and statistics
  • Enhanced Examples: Added 07_batch.exs and 08_query.exs with comprehensive batch and query demonstrations
  • Objects API Payload Builder: Type-safe payload construction in WeaviateEx.Objects.Payload
  • Collections API Extensions: Multi-tenancy support with set_multi_tenancy/2, improved tenant management
  • Development Scripts: scripts/weaviate-stack.sh wrapper for unified stack management
  • CI/Weaviate Infrastructure: Complete Docker Compose setup under ci/weaviate/ with helper scripts
  • Documentation: Extensive planning docs in docs/20251019/ covering essential scope, schema, queries, operations

Changed

  • Examples Overhaul: All 8 examples updated with improved error handling, cleaner output, and better demonstrations
  • README Improvements: Expanded documentation with embedded mode, Mix tasks, Docker management, and comprehensive usage guides
  • Mix Tasks Refactored: Cleaner implementation using WeaviateEx.DevSupport.Compose module for shared logic
  • Test Coverage: Added tests for batch operations, collections API, and data operations

Fixed

  • Example helper module visibility and formatting
  • Collections API tenant operations
  • Batch summary statistics and error reporting

[0.1.1] - 2025-10-16

Changed

  • Refactored HTTP client implementation into Protocol.HTTP.Client for better protocol-based architecture
  • Removed old HTTPClient and HTTPClient.Finch modules in favor of protocol-based implementation
  • Updated all examples to use cleaner ExampleHelper patterns with proper module qualification
  • Enhanced example output formatting and error handling
  • Fixed docker-compose port mapping (40051:50051 for gRPC)
  • Improved test cleanup and formatting across all test files

Added

  • Added vector support to data examples
  • Enhanced Protocol.HTTP.Client with comprehensive error handling
  • Added better response parsing and authentication header support

Fixed

  • Fixed example helper function visibility (made ANSI color helpers public)
  • Fixed client initialization to properly use protocol implementation
  • Improved error messages and debugging output

[0.1.0] - 2025-10-16

Added

  • Initial release