HuggingfaceClient.Inference.Provider behaviour (huggingface_client v0.1.0)

Copy Markdown View Source

Behaviour that every inference provider module must implement.

Mirrors the TaskProviderHelper abstract class from @huggingface/inference.

Provider anatomy

Each provider is responsible for:

  1. make_route/2 – Build the URL path segment (e.g. "v1/chat/completions").
  2. prepare_payload/2 – Transform unified task args into provider-specific body.
  3. get_response/3 – Validate and normalise the raw JSON response.
  4. prepare_headers/3 – Build auth/content-type headers.
  5. make_base_url/2 – Override to route through HF router vs direct provider.

Auth method

The :auth_method key in url_params / header_params can be:

  • :hf_token – token starts with "hf_", routed through HF router
  • :provider_key – bare API key, sent directly to the provider
  • :credentials_include – browser cookie auth (no explicit token)
  • :none – unauthenticated / public endpoints

Summary

Types

auth_method()
body_params()
header_params()
url_params()

Callbacks

client_side_routing_only?()

When true, the provider only supports direct API key auth. hf_token auth will raise an InputError.

get_response(response, opts)

Validates and normalises the raw decoded JSON response.

make_base_url(url_params)

Override to customise the base URL (e.g. HF router path for HF-hosted models).

make_route(url_params)

Returns the path segment appended to the base URL.

prepare_headers(header_params, binary?)

Builds request headers. binary? is true when the body is raw bytes.

prepare_payload(body_params)

Transforms unified args into the provider-specific request body.

provider_id()

Returns the provider identifier atom (e.g. :groq, :hf_inference). Used by the routing layer.

Types

auth_method()

@type auth_method() :: :hf_token | :provider_key | :credentials_include | :none

body_params()

@type body_params() :: %{
  args: map(),
  model: String.t(),
  task: String.t() | nil,
  mapping: map() | nil,
  output_type: atom() | nil
}

header_params()

@type header_params() :: %{access_token: String.t() | nil, auth_method: auth_method()}

url_params()

@type url_params() :: %{
  model: String.t(),
  task: String.t() | nil,
  auth_method: auth_method(),
  url_transform: (String.t() -> String.t()) | nil
}

Callbacks

client_side_routing_only?()

@callback client_side_routing_only?() :: boolean()

When true, the provider only supports direct API key auth. hf_token auth will raise an InputError.

get_response(response, opts)

@callback get_response(response :: term(), opts :: map()) ::
  {:ok, term()} | {:error, HuggingfaceClient.Error.ProviderOutputError.t()}

Validates and normalises the raw decoded JSON response.

Returns {:ok, normalised} or raises a ProviderOutputError.

make_base_url(url_params)

(optional) 
@callback make_base_url(url_params()) :: String.t()

Override to customise the base URL (e.g. HF router path for HF-hosted models).

make_route(url_params)

@callback make_route(url_params()) :: String.t()

Returns the path segment appended to the base URL.

prepare_headers(header_params, binary?)

@callback prepare_headers(header_params(), binary? :: boolean()) :: map()

Builds request headers. binary? is true when the body is raw bytes.

prepare_payload(body_params)

@callback prepare_payload(body_params()) :: map()

Transforms unified args into the provider-specific request body.

provider_id()

@callback provider_id() :: String.t()

Returns the provider identifier atom (e.g. :groq, :hf_inference). Used by the routing layer.