ReqLLM.Streaming.FinchClient (ReqLLM v1.0.0)

View Source

Finch HTTP client for ReqLLM streaming operations.

This module handles the Finch HTTP transport layer for streaming requests, forwarding HTTP events to StreamServer for processing. It acts as a bridge between Finch's HTTP streaming and the StreamServer's event processing.

Responsibilities

  • Build Finch.Request using provider-specific stream attachment
  • Start supervised Task that calls Finch.stream/5 with callback
  • Forward all HTTP events to StreamServer via GenServer.call
  • Handle connection errors and forward to StreamServer
  • Return HTTPContext for fixture capture

HTTPContext

The HTTPContext struct provides minimal HTTP metadata needed for fixture capture and testing, replacing the more heavyweight Req.Request/Response structs used in non-streaming operations.

Provider Integration

Uses provider-specific attach_stream/4 callbacks to build streaming requests with proper authentication, headers, and request body formatting.

Summary

Functions

start_stream(provider_mod, model, context, opts, stream_server_pid, finch_name \\ ReqLLM.Finch)

Starts a streaming HTTP request and forwards events to StreamServer.

Functions

start_stream(provider_mod, model, context, opts, stream_server_pid, finch_name \\ ReqLLM.Finch)

@spec start_stream(
  module(),
  ReqLLM.Model.t(),
  ReqLLM.Context.t(),
  keyword(),
  pid(),
  atom()
) ::
  {:ok, pid(), ReqLLM.Streaming.Fixtures.HTTPContext.t(), any()}
  | {:error, term()}

Starts a streaming HTTP request and forwards events to StreamServer.

Parameters

  • provider_mod - The provider module (e.g., ReqLLM.Providers.OpenAI)
  • model - The ReqLLM.Model struct
  • context - The ReqLLM.Context with messages to stream
  • opts - Additional options for the request
  • stream_server_pid - PID of the StreamServer GenServer
  • finch_name - Finch process name (defaults to ReqLLM.Finch)

Returns

  • {:ok, task_pid, http_context, canonical_json} - Successfully started streaming task
  • {:error, reason} - Failed to start streaming

The returned task will handle the Finch.stream/5 call and forward all HTTP events to the StreamServer. The HTTPContext provides minimal metadata for fixture capture.