Tinkex.ServiceClient (Tinkex v0.3.4)

View Source

Entry point for Tinkex operations.

Starts a session via Tinkex.SessionManager, tracks sequencing counters, and spawns Training/Sampling clients under Tinkex.ClientSupervisor.

Summary

Functions

Returns a specification to start this module under a supervisor.

Create a training client from this ServiceClient.

Return a REST client for session and checkpoint management.

Create a sampling client from this ServiceClient.

Create a sampling client asynchronously.

Create a training client from a saved checkpoint path.

Create a training client from checkpoint asynchronously.

Create a training client from a saved checkpoint path, loading optimizer state.

Fetch server capabilities (supported models/features) via the Service API.

Start a ServiceClient process.

Return the telemetry reporter pid if backend telemetry is enabled.

Types

t()

@type t() :: pid()

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

create_lora_training_client(service_client, base_model, opts \\ [])

@spec create_lora_training_client(t(), String.t(), keyword()) ::
  {:ok, pid()} | {:error, term()}

Create a training client from this ServiceClient.

base_model is a required second argument specifying the base model name (e.g., "meta-llama/Llama-3.1-8B"). Pass call_timeout: :infinity (or a larger timeout in ms) via opts if model creation may take longer than the default 5000ms GenServer call timeout.

create_lora_training_client_async(service_client, base_model, opts \\ [])

@spec create_lora_training_client_async(t(), String.t(), keyword()) :: Task.t()

Create a LoRA training client asynchronously.

Returns a Task that resolves to {:ok, pid} or {:error, reason}.

create_rest_client(service_client)

@spec create_rest_client(t()) :: {:ok, Tinkex.RestClient.t()}

Return a REST client for session and checkpoint management.

create_sampling_client(service_client, opts \\ [])

@spec create_sampling_client(
  t(),
  keyword()
) :: {:ok, pid()} | {:error, term()}

Create a sampling client from this ServiceClient.

create_sampling_client_async(service_client, opts \\ [])

@spec create_sampling_client_async(
  t(),
  keyword()
) :: Task.t()

Create a sampling client asynchronously.

Returns a Task that resolves to {:ok, pid} or {:error, reason}.

Examples

task = ServiceClient.create_sampling_client_async(service_pid, base_model: "meta-llama/Llama-3.2-1B")
{:ok, sampling_pid} = Task.await(task)

create_training_client_from_state(service_client, path, opts \\ [])

@spec create_training_client_from_state(t(), String.t(), keyword()) ::
  {:ok, pid()} | {:error, term()}

Create a training client from a saved checkpoint path.

Uses checkpoint metadata to configure the client, then loads the weights. To include optimizer state, pass load_optimizer: true or use create_training_client_from_state_with_optimizer/3.

create_training_client_from_state_async(service_client, path, opts \\ [])

@spec create_training_client_from_state_async(t(), String.t(), keyword()) :: Task.t()

Create a training client from checkpoint asynchronously.

Returns a Task that resolves to {:ok, pid} or {:error, reason}.

create_training_client_from_state_with_optimizer(service_client, path, opts \\ [])

@spec create_training_client_from_state_with_optimizer(t(), String.t(), keyword()) ::
  {:ok, pid()} | {:error, term()}

Create a training client from a saved checkpoint path, loading optimizer state.

Convenience wrapper around create_training_client_from_state/3 that sets load_optimizer: true.

create_training_client_from_state_with_optimizer_async(service_client, path, opts \\ [])

@spec create_training_client_from_state_with_optimizer_async(
  t(),
  String.t(),
  keyword()
) :: Task.t()

Async variant of create_training_client_from_state_with_optimizer/3.

get_server_capabilities(service_client)

@spec get_server_capabilities(t()) ::
  {:ok, Tinkex.Types.GetServerCapabilitiesResponse.t()} | {:error, term()}

Fetch server capabilities (supported models/features) via the Service API.

get_server_capabilities_async(service_client)

@spec get_server_capabilities_async(t()) :: Task.t()

Async helper for get_server_capabilities/1.

get_telemetry(server)

start_link(opts \\ [])

@spec start_link(keyword()) :: GenServer.on_start()

Start a ServiceClient process.

Accepts :config (Tinkex.Config.t()) and optional client modules via :training_client_module / :sampling_client_module.

telemetry_reporter(service_client)

@spec telemetry_reporter(t()) :: {:ok, pid()} | {:error, :disabled}

Return the telemetry reporter pid if backend telemetry is enabled.