HuggingfaceClient.Hub.Snapshots (huggingface_client v0.1.0)

Cache path utilities for HuggingFace Hub snapshots.

Mirrors the Python huggingface_hub.snapshot_download cache layout so that cached files produced by either library are interoperable.

Cache layout

$HF_HOME/hub/
  models--gpt2/
    refs/
      main
    blobs/
      <sha256>
    snapshots/
      <commit_sha>/
        config.json -> ../../blobs/<sha256>

Summary

Types

repo_type()

Functions

default_cache_dir()

Returns the default root cache directory for Hub snapshots.

repo_folder_name(repo_id, repo_type)

Returns the folder name for a given repo_id and repo_type.

repo_path(repo_id, repo_type, cache_dir \\ nil)

Returns the full on-disk path for a cached repo's snapshot root.

snapshot_path(repo_id, repo_type, revision, cache_dir \\ nil)

Returns the path for a specific snapshot revision of a repo.

Types

repo_type()

@type repo_type() :: :model | :dataset | :space

Functions

default_cache_dir()

@spec default_cache_dir() :: String.t()

Returns the default root cache directory for Hub snapshots.

Equivalent to $HF_HUB_CACHE, defaulting to ~/.cache/huggingface/hub.

repo_folder_name(repo_id, repo_type)

@spec repo_folder_name(String.t(), repo_type()) :: String.t()

Returns the folder name for a given repo_id and repo_type.

Replaces / with -- in the repo ID, matching the Python implementation.

Examples

iex> HuggingfaceClient.Hub.Snapshots.repo_folder_name("gpt2", :model)
"models--gpt2"

iex> HuggingfaceClient.Hub.Snapshots.repo_folder_name("meta-llama/Llama-3.1-8B-Instruct", :model)
"models--meta-llama--Llama-3.1-8B-Instruct"

iex> HuggingfaceClient.Hub.Snapshots.repo_folder_name("rajpurkar/squad", :dataset)
"datasets--rajpurkar--squad"

repo_path(repo_id, repo_type, cache_dir \\ nil)

@spec repo_path(String.t(), repo_type(), String.t() | nil) :: String.t()

Returns the full on-disk path for a cached repo's snapshot root.

~/.cache/huggingface/hub/models--gpt2/snapshots

snapshot_path(repo_id, repo_type, revision, cache_dir \\ nil)

@spec snapshot_path(String.t(), repo_type(), String.t(), String.t() | nil) ::
  String.t()

Returns the path for a specific snapshot revision of a repo.

~/.cache/huggingface/hub/models--gpt2/snapshots/<revision>