Gemini.RateLimiter.ConcurrencyGate (GeminiEx v0.9.0)

Copy Markdown View Source

Per-model concurrency gating using semaphore-like permits.

Throttles request bursts by limiting concurrent requests per model. Supports adaptive mode that adjusts concurrency based on 429 responses.

Features

  • Configurable per-model concurrency limits
  • Adaptive mode: starts low, raises until 429, then backs off
  • Non-blocking mode support for immediate returns
  • ETS-based permit tracking for cross-process visibility

Summary

Types

model_key()
permit_state()

Functions

acquire(model, config)

Acquire a permit for the given model.

available_permits(model, config)

Get the number of available permits for a model.

get_state(model)

Get current permit state for a model.

handle_holder_down(model, holder_pid)
init()

Initialize the ETS table for permit tracking.

release(model)

Release a permit for the given model.

reset_all()

Reset all state (useful for testing).

signal_429(model, config)

Signal that a 429 was received for adaptive backoff.

signal_success(model, config)

Signal that a request succeeded for adaptive raise.

Types

model_key()

@type model_key() :: String.t()

permit_state()

@type permit_state() :: %{
  current: non_neg_integer(),
  max: pos_integer(),
  adaptive_max: pos_integer() | nil,
  waiting: [pid()],
  holders: %{required(pid()) => {non_neg_integer(), pid()}}
}

Functions

acquire(model, config)

@spec acquire(model_key(), Gemini.RateLimiter.Config.t()) :: :ok | {:error, atom()}

Acquire a permit for the given model.

Returns immediately if a permit is available. If no permit is available:

  • In blocking mode: waits until a permit becomes available
  • In non-blocking mode: returns {:error, :no_permit_available}

Parameters

  • model - Model name
  • config - Rate limiter configuration

Returns

  • :ok - Permit acquired
  • {:error, :no_permit_available} - No permit available (non-blocking mode)
  • {:error, :concurrency_disabled} - Concurrency gating is disabled

available_permits(model, config)

@spec available_permits(model_key(), Gemini.RateLimiter.Config.t()) ::
  non_neg_integer()

Get the number of available permits for a model.

get_state(model)

@spec get_state(model_key()) :: permit_state() | nil

Get current permit state for a model.

handle_holder_down(model, holder_pid)

init()

@spec init() :: :ok

Initialize the ETS table for permit tracking.

Called automatically when the RateLimitManager starts, but also lazily initialized on first access to support direct calls without the supervisor running.

release(model)

@spec release(model_key()) :: :ok

Release a permit for the given model.

Called after a request completes (success or failure).

reset_all()

@spec reset_all() :: :ok

Reset all state (useful for testing).

signal_429(model, config)

@spec signal_429(model_key(), Gemini.RateLimiter.Config.t()) :: :ok

Signal that a 429 was received for adaptive backoff.

In adaptive mode, reduces the effective max concurrency.

signal_success(model, config)

@spec signal_success(model_key(), Gemini.RateLimiter.Config.t()) :: :ok

Signal that a request succeeded for adaptive raise.

In adaptive mode, gradually increases concurrency up to the ceiling.