Mistral.Cache (Mistral v0.5.0)

Optional response caching for Mistral API requests.

Implements a Req plugin that caches successful responses in ETS to reduce redundant API calls. Caching is applied selectively — only deterministic, non-streaming endpoints are cached.

Cached endpoints

  • GET requests (models, files, agents)
  • Deterministic POST endpoints: /embeddings, /classifications, /chat/classifications, /moderations, /chat/moderations, /ocr

Skipped requests

  • Streaming requests (where into option is set)
  • Non-deterministic POST endpoints (/chat/completions, /fim/completions, /agents/completions)
  • Mutation endpoints (PUT, PATCH, DELETE, non-cacheable POST)
  • Error responses (non-2xx)

Usage

Caching is enabled through Mistral.init/2:

client = Mistral.init("api_key", cache: true)
client = Mistral.init("api_key", cache: true, cache_ttl: 600_000)
client = Mistral.init("api_key", cache: true, cache_table: :my_cache)

TTL

Entries expire lazily on read. Use sweep/2 for bulk cleanup of expired entries. Default TTL is 5 minutes (300,000 ms).

ETS table

The cache uses a named ETS table (default :mistral_cache) with read_concurrency and write_concurrency enabled. The table is created lazily on first use and owned by a long-lived process.

Summary

Functions

attach(request, opts \\ [])

Attaches the cache plugin to a Req.Request.

clear(table \\ :mistral_cache)

Deletes all entries from the cache table.

invalidate(pattern, table \\ :mistral_cache)

Deletes cache entries whose URL contains the given pattern substring.

stats(table \\ :mistral_cache)

Returns cache statistics.

sweep(ttl \\ 300_000, table \\ :mistral_cache)

Removes expired entries from the cache. Returns the number of entries deleted.

Functions

attach(request, opts \\ [])

@spec attach(
  Req.Request.t(),
  keyword()
) :: Req.Request.t()

Attaches the cache plugin to a Req.Request.

Options

  • :cache_ttl - Time-to-live in milliseconds. Defaults to 300000.
  • :cache_table - ETS table name. Defaults to :mistral_cache.

clear(table \\ :mistral_cache)

@spec clear(atom()) :: :ok

Deletes all entries from the cache table.

invalidate(pattern, table \\ :mistral_cache)

@spec invalidate(String.t(), atom()) :: non_neg_integer()

Deletes cache entries whose URL contains the given pattern substring.

Returns the number of entries deleted.

stats(table \\ :mistral_cache)

@spec stats(atom()) :: %{size: non_neg_integer(), memory_bytes: non_neg_integer()}

Returns cache statistics.

Return value

%{size: non_neg_integer(), memory_bytes: non_neg_integer()}

sweep(ttl \\ 300_000, table \\ :mistral_cache)

@spec sweep(non_neg_integer(), atom()) :: non_neg_integer()

Removes expired entries from the cache. Returns the number of entries deleted.