ReqLLM.Application (ReqLLM v1.0.0)

View Source

Application supervisor for ReqLLM.

Starts and supervises the Finch instance used for streaming LLM APIs. Provides optimized connection pools per provider with sensible defaults that can be overridden via application configuration.

Summary

Functions

finch_name()

Gets the default Finch name used by ReqLLM for streaming operations.

get_finch_config()

Gets the Finch configuration from application environment with unified pool defaults.

Functions

finch_name()

@spec finch_name() :: atom()

Gets the default Finch name used by ReqLLM for streaming operations.

get_finch_config()

@spec get_finch_config() :: keyword()

Gets the Finch configuration from application environment with unified pool defaults.

ReqLLM normalizes all providers through a single connection pool, making it as easy as changing the model spec to switch providers.

Users can override pool configurations by setting:

config :req_llm,
  finch: [
    name: ReqLLM.Finch,
    pools: %{
      :default => [protocols: [:http2, :http1], size: 1, count: 16]
    }
  ]