A token-bucket rate limiter for controlling request frequency.

A rate limiter wraps calls to a downstream service and limits how many can execute per second. When the token bucket is empty, callers are rejected immediately with a retry_after hint telling them when to try again.

Unlike Resiliency.Bulkhead, which controls concurrency (how many calls can be in-flight at once), the rate limiter controls frequency (requests per second). They complement each other and can be composed.

When to use

• You need to stay under a rate limit imposed by an external API or service (e.g., "100 requests per second").
• You want to smooth out bursty traffic to protect a downstream system.
• You need weighted requests — expensive operations can consume more tokens than lightweight ones.

Quick start

# 1. Add to your supervision tree
children = [
  {Resiliency.RateLimiter, name: MyApp.ApiRateLimiter, rate: 100.0, burst_size: 10}
]
Supervisor.start_link(children, strategy: :one_for_one)

# 2. Use it
case Resiliency.RateLimiter.call(MyApp.ApiRateLimiter, fn -> HttpClient.get(url) end) do
  {:ok, response} -> handle_response(response)
  {:error, {:rate_limited, retry_after_ms}} -> {:error, {:overloaded, retry_after_ms}}
  {:error, reason} -> {:error, reason}
end

How it works

The rate limiter uses a token bucket algorithm with lazy refill — there is no background timer or process. Instead, tokens are refilled on each call based on how much time has elapsed since the last refill.

• The bucket starts full at burst_size tokens.
• Tokens refill at rate tokens per second (lazily on each call).
• Each call consumes weight tokens (default 1).
• If enough tokens are available, the call proceeds immediately.
• If not, the call is rejected with a retry_after_ms hint.

Token acquisition uses a lock-free ETS CAS loop — callers read the bucket state, compute the refill, and atomically update the row via :ets.select_replace/2. On conflict (two callers racing), the loser retries immediately with the freshly-written state. This means:

• No GenServer message is sent on the hot call/2,3 path.
• Throughput scales with the number of schedulers, not a single mailbox.
• A lightweight GenServer owns the ETS table and handles reset/1.
• The protected function runs in the caller's process; a crash there does not affect the rate limiter state.

Return values

Note: bare values returned by fun (not wrapped in {:ok, _} or {:error, _}) are automatically wrapped in {:ok, value}. This follows the BackoffRetry convention and differs from Resiliency.Bulkhead which returns raw results.

Error handling

If the function raises, exits, or throws, the error is caught and returned to the caller. The token consumed for the call is not refunded, as the call did execute:

{:error, {%RuntimeError{message: "boom"}, _stacktrace}} =
  Resiliency.RateLimiter.call(MyApp.ApiRateLimiter, fn -> raise "boom" end)

Telemetry

All events are emitted in the caller's process. See Resiliency.Telemetry for the complete event catalogue.

[:resiliency, :rate_limiter, :call, :start]

Emitted at the beginning of every call/2,3 invocation, before the token acquisition attempt.

Measurements

Metadata

[:resiliency, :rate_limiter, :call, :rejected]

Emitted when the token bucket is empty and the call is rejected without executing the function. Always followed immediately by a :stop event.

Measurements

Metadata

[:resiliency, :rate_limiter, :call, :stop]

Emitted after every call/2,3 completes — whether rejected, successful, or failed.

Measurements

Metadata