Changelog

View Source

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

0.4.0 - 2025-10-13

Added

  • Automatic Redaction System - ExVCR-style redact_cassette macro for programmatic data redaction
    • redact_cassette Macro - Wrap API calls to automatically redact sensitive data during recording and replay
    • User-Defined Redaction Functions - Custom functions for response_body_json, response_body_raw, request_headers, response_headers, and url redaction
    • JSON-Aware Redaction - Automatic JSON detection and encoding/decoding using configurable Reqord.JSON behavior
    • Named Redactors - Define reusable redaction configurations in application config and reference by name
    • Flexible Function Support - Inline functions, named functions, module/function tuples, and config-based redactors
    • Bidirectional Application - Redaction applied during both recording (to cassettes) and replay (to test responses) for consistency
    • Content-Type Detection - Smart detection of JSON vs binary content for appropriate redaction handling
  • Object and Streaming Support - Comprehensive support for binary data and streaming responses
    • Smart Content Detection - Automatic detection of binary vs text content based on Content-Type headers and heuristics
    • External Object Storage - Large binary objects stored externally to prevent JSONL bloat
    • Streaming Response Support - Capture and replay of streaming responses (SSE, chunked transfer)
    • Configurable Storage Thresholds - Control when to use external vs inline storage
    • Enhanced CassetteEntry.Response - New fields for body_encoding, body_external_ref, and stream_metadata
    • Pluggable Storage Extensions - Extended Storage.Behavior with object and stream storage methods
  • Flexible Cassette Organization - Multiple strategies for organizing cassettes
    • Explicit Path Override - Use :vcr_path tag to specify exact cassette path per test
    • Named Path Builders - Define reusable path builders in config and reference by name in test modules
    • Global Path Builder - Configure a single path builder function for all tests
    • Macro Context Support - New Reqord.Case.set_cassette_context/1 for passing compile-time variables to path builders
    • Priority System - Clear precedence: vcr_path > named builder > global builder > vcr tag > default
    • Documentation - Comprehensive guides in docs/CASSETTE_ORGANIZATION.md and docs/MACRO_SUPPORT.md
  • Manual Cassette Editing - New mix reqord.edit task for redacting sensitive data
    • Opens cassettes in your editor with automatic JSONL parsing and formatting
    • Validates JSON structure after editing before saving
    • Filter by entry index or URL pattern
    • Useful for redacting emails, account IDs, and other PII from response bodies
  • Improved Mix Task Path Handling - All Mix tasks now support flexible path resolution
    • Accept short names (relative to cassette dir), relative paths, and absolute paths
    • Consistent with mix test command usage
    • Shared Reqord.Tasks.Helpers module reduces code duplication across tasks

Changed

  • Enhanced Manual Redaction - Improved mix reqord.edit task to work seamlessly with automatic redaction system
  • Enhanced Response Creation - New CassetteEntry.Response.new_with_raw_body/3 for automatic encoding detection
  • Improved Replay System - Enhanced body loading with support for external storage and streaming content
  • Extended Configuration - New config options for max_inline_size, object_directory, binary_storage, and stream_speed
  • Documentation Organization - Moved guide documents to docs/ directory and added to hex docs
  • README Simplification - Streamlined README to focus on quick start and core features
    • Split detailed content into focused guides (Getting Started, Security, Advanced Configuration)
    • Reduced from 691 to ~250 lines for better new-user experience
    • Added clear navigation to comprehensive documentation
  • New Documentation Guides
    • Getting Started Guide - Simple, beginner-friendly introduction with security basics
    • Security Guide - Comprehensive redaction documentation with best practices, pre-commit hooks, and emergency procedures
    • Advanced Configuration Guide - Detailed feature documentation for request matching, binary data, streaming, and performance tuning

0.3.0 - 2025-10-05

Added

  • NEW ARCHITECTURE: Timestamp-based chronological ordering - Complete redesign to solve request ordering issues
    • Microsecond precision timestamps - All cassette entries now include recorded_at field with microsecond timestamps
    • Async CassetteWriter GenServer - Non-blocking writes with batching and automatic timestamp sorting
    • Streaming CassetteReader - Memory-efficient reading with chronological ordering by timestamp
    • Pluggable Storage Backend - New Reqord.Storage.Behavior interface for future S3/Redis support
    • FileSystem Storage Backend - Optimized JSONL file operations with atomic writes and streaming reads
    • Application Supervision - CassetteWriter automatically starts with Reqord application
    • Enhanced Test Flushing - Automatic cassette flushing on test completion via Reqord.cleanup/1
  • Pure Sequential Streaming - Revolutionary performance improvement for cassette replay
    • No search operations - Direct O(1) access instead of O(n) searching through entries
    • Sequential verification - Takes next entry and verifies match instead of searching for matches
    • Optimized common case - Fast path for default [:method, :uri] matching used in 90%+ of cases

Fixed

  • CRITICAL: POST-DELETE lifecycle ordering - Solved concurrent request recording order issues
    • Problem: Concurrent requests (e.g., parallel POST-DELETE lifecycles) were recorded in completion order, not initiation order
    • Solution: Timestamp-based recording ensures chronological replay even when requests complete out of order
    • Impact: Eliminates ID mismatch errors in cassette replay for concurrent scenarios
    • Example: POST (user creation) → DELETE (user deletion) lifecycles now maintain correct order during replay
  • CRITICAL: Concurrent request recording in :all mode - Fixed issue where HTTP requests made from spawned processes weren't being recorded
    • Problem: Reqord used process dictionary to accumulate requests in :all mode, but spawned processes (Task.async, etc.) don't inherit parent's process dictionary
    • Solution: Replaced process dictionary with GenServer-based state management following ExVCR's proven architecture pattern
    • Impact: Concurrent HTTP requests from Task.async and other spawned processes are now properly recorded in :all mode
    • Scope: Only affects :all mode - other modes (:once, :new_episodes, :none) unchanged and working correctly

Changed

  • BREAKING: Cassette format change - All cassettes now require timestamps
    • Migration Required: Existing cassettes without timestamps will not load
    • Solution: Regenerate all cassettes using REQORD=all mix test
    • Benefit: Enables chronological replay ordering and future extensibility
  • BREAKING: Removed legacy timestamp compatibility - Clean implementation without backward compatibility
    • No automatic timestamp addition for legacy entries
    • Simplified codebase with consistent timestamp requirements
    • Better error messages for invalid cassette entries
  • BREAKING: Sequential replay is now the only strategy - Removed complex dual matching approaches
    • Eliminated: Search-based "last match wins" strategy that was slower and more error-prone
    • Unified: Single sequential streaming approach for all scenarios
    • Improved: Better error messages with SequenceMismatchError showing exact position and expected vs actual requests
    • Simplified: Much cleaner codebase with reduced cognitive overhead

Removed

  • Legacy cassette support - No longer supports cassettes without timestamps for cleaner architecture
  • Search-based matching - Removed complex search algorithms in favor of simple sequential access
  • Duplicate code - Eliminated duplicate put_headers functions across modules

Performance

  • MAJOR: O(1) cassette replay - Eliminated O(n) search operations for massive performance improvement on large cassettes
  • Async writes - Non-blocking cassette writes during test execution
  • Streaming operations - Memory-efficient reading for large cassette files
  • Batched I/O - Reduced file system operations through intelligent batching
  • Timestamp sorting - Automatic chronological ordering without manual intervention
  • Optimized matching - Inlined fast path for most common matching scenarios

0.2.2 - 2025-10-03

Fixed

  • CRITICAL: Multiple requests in :all mode - Fixed :all mode to properly handle multiple requests
    • In :all mode, each request now accumulates and replaces the entire test cassette with all requests
    • This ensures all requests in a test are recorded while never clearing cassettes from other tests
    • Each test gets a fresh cassette containing only its requests when using REQORD=all
    • Fixes the issue where running REQORD=all mix test specific_test.exs would inappropriately clear cassettes

0.2.1 - 2025-10-03

Fixed

  • CRITICAL: Record mode behavior - Fixed :all mode to follow Ruby VCR behavior
    • :all mode now always replaces the entire cassette (instead of appending)
    • This fixes the critical issue where rerecording would mix old broken requests with new fixed ones
    • Simplified implementation by removing complex session tracking in favor of Ruby VCR's straightforward approach
    • :all mode now never replays - always goes live and records fresh responses
    • Ensures that REQORD=all provides clean, predictable cassette replacement behavior
  • Code quality - Fixed all test warnings and Credo issues
    • Resolved unused variable warnings in test files
    • Fixed alias ordering and unused alias warnings
    • Refactored complex functions to reduce cyclomatic complexity
    • Improved code readability and maintainability

Added

  • Comprehensive test coverage - Extensive edge case testing across all modules
    • Redactor tests for extreme token lengths (1-10000 characters) and boundary cases
    • URL normalization edge cases including malformed URLs and unicode handling
    • Base64 encoding/decoding stress tests with large data and concurrent operations
    • Cassette file format edge cases (mixed line endings, BOM markers, malformed JSON)
    • HTTP request/response validation for unusual methods and status codes
    • Record mode integration tests simulating real-world broken→fixed workflows
    • Last-match-wins replay strategy tests for handling mixed cassette scenarios
    • Large file handling and concurrent access patterns
    • Memory and performance considerations for encoding operations
  • Documentation improvements - Added hex package badges to README

Changed

  • Simplified architecture - Removed Session module complexity
    • Eliminated Agent-based session tracking that was causing confusion
    • Record logic now follows simple Ruby VCR patterns without stateful tracking
    • Cleaner, more predictable behavior that matches developer expectations
    • Reduced cognitive overhead for understanding and debugging record modes

0.2.0 - 2025-10-02

Added

  • Pluggable JSON system - Support for custom JSON libraries via Reqord.JSON behavior
    • Reqord.JSON.Jason adapter (default) with runtime availability checks
    • Reqord.JSON.Poison adapter with graceful fallback when not available
    • Application config: config :reqord, :json_library, MyAdapter
    • Comprehensive adapter test suite with real cassette creation/loading
  • CassetteEntry struct - Type-safe cassette data modeling with validation
    • Nested Reqord.CassetteEntry.Request and Response structs
    • Built-in validation with helpful error messages
    • Helper functions: new/2, from_raw/1, to_map/1, validate/1
  • Test utilities - Shared helpers to reduce test code duplication
    • Reqord.TestHelpers.with_module/3 for conditional module tests
    • Reqord.TestHelpers.with_config/4 for application config setup/teardown
    • Reqord.TestHelpers.with_module_and_config/6 combining both patterns
  • Development tools - Mix task for code quality enforcement
    • mix precommit alias running format, credo, dialyzer, and tests
  • Configurable settings - Made hard-coded values configurable for flexibility
    • Reqord.Config module for centralized configuration management
    • Configurable cassette directory: config :reqord, :cassette_dir, "custom/path"
    • Configurable auth parameters: config :reqord, :auth_params, ~w[token my_token]
    • Configurable auth headers: config :reqord, :auth_headers, ~w[authorization x-my-auth]
    • Configurable volatile headers: config :reqord, :volatile_headers, ~w[date x-trace-id]
    • Configuration validation with helpful error messages
    • Comprehensive test coverage for all configuration options

Changed

  • Improved error handling throughout the codebase
    • Replaced bare rescue _ -> body clauses with specific exception handling
    • Added logging for JSON decoding failures and network errors during recording
    • Consistent use of case statements instead of exception-prone ! functions
  • Better test organization - Separated JSONL format tests into dedicated file
    • test/reqord/jsonl_test.exs for JSONL-specific functionality
    • test/reqord/json/ directory for JSON adapter tests
    • Improved test cleanup preserving fixture files
  • Optional dependencies - Made JSON libraries optional to reduce bloat
    • Jason and Poison marked as optional: true in mix.exs
    • Runtime checks with helpful error messages when libraries are missing

Fixed

  • Error handling security - Prevented silent failures that could mask issues
  • Test reliability - Fixed cassette cleanup logic to preserve fixture directories
  • Code quality - Resolved Credo warnings for negated conditions and formatting
  • Hard-coded values - Eliminated hard-coded cassette directory, auth parameters, and volatile headers
    • All previously hard-coded values are now configurable via application config
    • Maintains backward compatibility with sensible defaults

0.1.0 - 2025-10-02

Added

  • Initial release of Reqord
  • Core Reqord module with install!/1 for setting up VCR stubs
  • Ruby VCR-style record modes: :once, :new_episodes, :all, :none
    • :once - Strict replay, raise on new requests (default)
    • :new_episodes - Replay existing, record new requests (append mode)
    • :all - Always re-record everything, ignore existing cassette
    • :none - Never record, never hit network (must have complete cassette)
  • Reqord.Case ExUnit case template for automatic cassette management
  • Smart request matching based on method, normalized URL, and body hash
  • Automatic redaction of sensitive headers (authorization)
  • Automatic redaction of auth query parameters (token, apikey, api_key)
  • Automatic removal of volatile response headers (date, server, set-cookie, etc.)
  • JSONL cassette format stored in test/support/cassettes/
  • Query parameter normalization (lexicographic sorting, auth param removal)
  • Body hash differentiation for POST/PUT/PATCH requests
  • Support for spawned processes via allow/3 helper
  • Multiple configuration options for record mode:
    • Environment variable via REQORD
    • Application config via :reqord, :default_mode
    • Per-test override via @tag vcr_mode: :mode
  • Automatic cassette naming based on test module and test name
  • Custom cassette naming via @tag vcr: "custom_name"
  • Custom stub name override via @tag req_stub_name: MyStub
  • Integration with Req.Test for zero application code changes
  • Comprehensive test suite covering all modes and features
  • Detailed README with setup, usage examples, and troubleshooting
  • Mix tasks for cassette management:
  • Flexible request matching system:
    • Built-in matchers: :method, :uri, :host, :path, :headers, :body
    • Custom matcher registration via register_matcher/2
    • Default matching on [:method, :uri]
    • Per-test matcher override via @tag match_on: [...matchers]
    • Application config for default matchers
  • Test API application (test_api/) for demonstrating Reqord:
    • Simple REST API with authentication
    • Multiple routes (GET /api/users, GET /api/users/:id, POST /api/users)
    • Fake Bearer token authentication
    • Example tests showing real-world usage
  • Automated cassette recording script (scripts/record_cassettes.sh):
    • Automatically starts test API server
    • Records all example cassettes
    • Stops server when complete
  • Comprehensive secret redaction system (Reqord.Redactor):
    • CRITICAL SECURITY: Ensures secrets never get committed to git cassettes
    • Built-in redaction for auth headers, query parameters, response bodies
    • VCR-style configurable filters for app-specific secrets
    • Multi-layer protection: headers → query params → JSON keys → pattern matching
    • Automatic redaction of Bearer tokens, API keys, long alphanumeric strings
    • Support for GitHub tokens, Stripe keys, UUIDs, and custom patterns