View Source Agens.Serving.Config (agens v0.1.3)

The Config struct represents the configuration for a Serving process.

Fields

  • :name - The unique name for the Serving process.
  • :serving - The Nx.Serving struct or GenServer module for the Agens.Serving.
  • :prefixes - An Agens.Prefixes struct of custom prompt prefixes. If nil, default prompt prefixes will be used instead. Default prompt prefixes can also be overridden by using the prefixes options in Agens.Supervisor.
  • :finalize - A function that accepts the prepared prompt (including any applied prefixes) and returns a modified version of the prompt. Useful for wrapping the prompt or applying final processing before sending to the LM for inference. If nil, the prepared prompt will be used as-is.
  • :args - Additional arguments to be passed to the Nx.Serving or GenServer module. See the Nx.Serving or GenServer documentation for more information.

Summary

Types

t()

Types

Link to this type

t()

View Source 
@type t() :: %Agens.Serving.Config{
  args: keyword(),
  finalize: (String.t() -> String.t()) | nil,
  name: atom(),
  prefixes: Agens.Prefixes.t() | nil,
  serving: Nx.Serving.t() | module()
}