View Source Agens.Serving (agens v0.1.3)
The Serving module provides functions for starting, stopping and running Servings.
Agens.Serving accepts a GenServer module or Nx.Serving struct for processing messages.
Agens.Serving is decoupled from Agens.Agent in order to reuse a single LM across multiple agents. In most cases, however, you will only need to start one text generation serving to be used by most, if not all, agents.
In some cases, you may have additional servings for more specific use cases such as image generation, speech recognition, etc.
Servings were built with the Bumblebee library in mind, as well as Nx.Serving. GenServer is supported for working with LM APIs instead, which may be more cost effective and easier to get started with.
Summary
Functions
Retrieves the Serving configuration by Serving name or pid.
Executes an Agens.Message against an Agens.Serving
Starts an Agens.Serving process
Stops an Agens.Serving process
Functions
@spec get_config(atom() | pid()) :: {:ok, Agens.Serving.Config.t()} | {:error, :serving_not_found}
Retrieves the Serving configuration by Serving name or pid.
@spec run(Agens.Message.t()) :: String.t() | {:error, :serving_not_found}
Executes an Agens.Message against an Agens.Serving
@spec start(Agens.Serving.Config.t()) :: {:ok, pid()} | {:error, term()}
Starts an Agens.Serving process
@spec stop(atom()) :: :ok | {:error, :serving_not_found}
Stops an Agens.Serving process