HfHub (HfHub v0.1.3)

View Source

Elixir client for HuggingFace Hub.

HfHub provides a comprehensive interface to the HuggingFace Hub API, enabling Elixir applications to access models, datasets, and spaces.

Features

  • Hub API Client — Fetch metadata for models, datasets, and spaces
  • File Downloads — Stream files from HuggingFace repositories with resume support
  • Smart Caching — Local file caching with LRU eviction and integrity checks
  • Filesystem Utilities — Manage local HuggingFace cache directory structure
  • Authentication — Token-based authentication for private repositories

Quick Start

# Get model information
{:ok, model_info} = HfHub.Api.model_info("bert-base-uncased")

# Download a model file
{:ok, path} = HfHub.Download.hf_hub_download(
  repo_id: "bert-base-uncased",
  filename: "config.json",
  repo_type: :model
)

# Check cache
cached? = HfHub.Cache.cached?(
  repo_id: "bert-base-uncased",
  filename: "config.json"
)

Configuration

Configure in config/config.exs:

config :hf_hub,
  token: System.get_env("HF_TOKEN"),
  cache_dir: Path.expand("~/.cache/huggingface"),
  endpoint: "https://huggingface.co"

Modules

Summary

Types

A repository reference (Bumblebee-compatible).

Types

filename()

@type filename() :: String.t()

repo_id()

@type repo_id() :: String.t()

repo_type()

@type repo_type() :: :model | :dataset | :space

repository()

@type repository() :: HfHub.Repository.t()

A repository reference (Bumblebee-compatible).

Can be either:

  • {:hf, repository_id} - HuggingFace repository
  • {:hf, repository_id, opts} - HuggingFace repository with options
  • {:local, directory} - Local directory

revision()

@type revision() :: String.t()

Functions

cached_download(url, opts \\ [])

See HfHub.Hub.cached_download/2.

file_listing_url(repository_id, subdir, revision)

See HfHub.Hub.file_listing_url/3.

file_url(repository_id, filename, revision)

See HfHub.Hub.file_url/3.

get_repo_files(repository)

See HfHub.RepoFiles.get_repo_files/1.