HuggingfaceClient.Inference.ProviderMapping (huggingface_client v0.1.0)

Copy Markdown View Source

ETS-backed GenServer that caches inferenceProviderMapping data fetched from the HuggingFace Hub API (GET /api/models/:model_id).

Responsibilities

  • Normalises both array-of-objects and legacy map-of-objects API formats.
  • Caches results in an ETS table so repeated calls for the same model skip the network round-trip.
  • Provides resolve_provider/3 to determine the effective provider string before a request is dispatched.

Summary

Functions

Returns a specification to start this module under a supervisor.

Clears the entire ETS cache.

Returns the provider mapping entry for {model_id, provider}.

Normalises a raw inferenceProviderMapping API value into a flat list of maps, each containing at least "provider", "hf_model_id", and "provider_id".

Resolves the effective provider string given the user-supplied provider, model_id, and optional endpoint_url.

Stores a list of normalised mapping entries into the ETS cache.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

clear_cache()

@spec clear_cache() :: :ok

Clears the entire ETS cache.

get_mapping(model_id, provider)

@spec get_mapping(String.t(), String.t()) :: {:ok, map()} | {:error, term()}

Returns the provider mapping entry for {model_id, provider}.

For provider == "auto" returns a synthetic mapping immediately without hitting the network. For other providers, checks the ETS cache first, then would fetch from the Hub (stubbed here — real fetch via Hub.Client is wired in HuggingfaceClient.Inference).

normalise_mapping(list, model_id)

@spec normalise_mapping(term(), String.t()) :: [map()]

Normalises a raw inferenceProviderMapping API value into a flat list of maps, each containing at least "provider", "hf_model_id", and "provider_id".

Handles:

  • nil / missing → []
  • Array format: [%{"provider" => "groq", ...}]
  • Legacy map format: %{"groq" => %{"providerId" => ...}, ...}

resolve_provider(provider, model_id, endpoint_url \\ nil)

@spec resolve_provider(String.t() | nil, String.t() | nil, String.t() | nil) ::
  {:ok, String.t()} | {:error, HuggingfaceClient.Error.InputError.t()}

Resolves the effective provider string given the user-supplied provider, model_id, and optional endpoint_url.

Decision matrix:

  • endpoint_url present → always "hf-inference" (direct endpoint, no routing)
  • explicit provider string (not "auto") → pass through unchanged
  • nil"auto"
  • "auto" with no model → {:error, InputError}
  • "auto" with model → "auto" (caller resolves later via get_mapping/2)

start_link(opts \\ [])

store_mappings(mappings)

@spec store_mappings([map()]) :: :ok

Stores a list of normalised mapping entries into the ETS cache.