HuggingfaceClient.Hub.Files (huggingface_client v0.1.0)

Copy Markdown View Source

File listing, existence checks, downloads, paths-info, and safetensors parsing.

Summary

Functions

Deletes an entire folder from a repository.

Downloads a file. See HuggingfaceClient.download_file/3.

Checks if a file exists. See HuggingfaceClient.file_exists/3.

Downloads a single file from a Hub repository to a local cache directory.

Returns the URL to download a file from the Hub.

Lists files in a repo. See HuggingfaceClient.list_files/2.

Parses safetensors metadata from a model repo.

Returns path info (metadata) for specific files in a repository.

Returns metadata for a list of paths. See HuggingfaceClient.paths_info/3.

Functions

delete_folder(repo_id, opts)

@spec delete_folder(
  String.t(),
  keyword()
) :: {:ok, map()} | {:error, Exception.t()}

Deletes an entire folder from a repository.

This creates a commit that removes all files under the given folder_path. Equivalent to Python's HfApi.delete_folder().

Options

  • :folder_path — path of the folder to delete (required)
  • :type:model, :dataset, :space (default: :model)
  • :commit_message — commit message
  • :revision — branch (default: "main")
  • :access_token

Example

{:ok, _} = HuggingfaceClient.Hub.Files.delete_folder(
  "my-org/my-model",
  folder_path: "old-checkpoints/",
  commit_message: "Remove old checkpoints",
  access_token: token
)

download(repo, path, opts)

@spec download(String.t(), String.t(), keyword()) ::
  {:ok, binary() | nil} | {:error, Exception.t()}

Downloads a file. See HuggingfaceClient.download_file/3.

exists?(repo, path, opts)

@spec exists?(String.t(), String.t(), keyword()) ::
  {:ok, boolean()} | {:error, Exception.t()}

Checks if a file exists. See HuggingfaceClient.file_exists/3.

hf_hub_download(repo_id, opts \\ [])

@spec hf_hub_download(
  String.t(),
  keyword()
) :: {:ok, String.t()} | {:error, Exception.t()}

Downloads a single file from a Hub repository to a local cache directory.

Equivalent to Python's hf_hub_download(). Downloads the file and caches it, returning the local path. The cache follows the standard HF cache layout.

Options

  • :filename — path of the file within the repo (required)
  • :revision — branch, tag, or commit SHA (default: "main")
  • :cache_dir — override the default cache directory
  • :type — repo type (default: :model)
  • :access_token

Example

{:ok, path} = HuggingfaceClient.hf_hub_download("gpt2",
  filename: "config.json"
)
IO.puts("Downloaded to: #{path}")

# From a dataset
{:ok, path} = HuggingfaceClient.hf_hub_download("rajpurkar/squad",
  filename: "squad.py",
  type: :dataset,
  revision: "main"
)

hf_hub_url(repo_id, opts \\ [])

@spec hf_hub_url(
  String.t(),
  keyword()
) :: String.t()

Returns the URL to download a file from the Hub.

Equivalent to Python's hf_hub_url().

Options

  • :filename — file path in the repo (required)
  • :revision — branch, tag, or SHA (default: "main")
  • :type — repo type (default: :model)

Example

url = HuggingfaceClient.hf_hub_url("gpt2", filename: "config.json")
# "https://huggingface.co/gpt2/resolve/main/config.json"

list(repo, opts)

@spec list(
  String.t(),
  keyword()
) :: Enumerable.t()

Lists files in a repo. See HuggingfaceClient.list_files/2.

parse_safetensors(repo, opts)

@spec parse_safetensors(
  String.t(),
  keyword()
) :: {:ok, map()} | {:error, Exception.t()}

Parses safetensors metadata from a model repo.

Supports:

  • model.safetensors (single-shard)
  • model.safetensors.index.json (multi-shard)

Returns

{:ok, %{
  "parameters" => %{"total" => integer, "by_dtype" => map},
  "dtype_distribution" => map,
  "shards" => [%{file: path, tensors: [...]}]
}}

paths_info(repo_id, opts)

@spec paths_info(
  String.t(),
  keyword()
) :: {:ok, [map()]} | {:error, Exception.t()}

Returns path info (metadata) for specific files in a repository.

Equivalent to Python's HfApi.get_paths_info().

Options

  • :paths — list of file paths (required)
  • :type — repo type
  • :revision — branch (default: "main")
  • :access_token

Example

{:ok, infos} = HuggingfaceClient.Hub.Files.paths_info(
  "gpt2",
  paths: ["config.json", "vocab.json"],
  access_token: token
)
Enum.each(infos, fn info ->
  IO.puts("#{info["rfilename"]}: #{info["size"]} bytes, sha=#{info["sha"]}")
end)

paths_info(repo, paths, opts)

@spec paths_info(String.t(), [String.t()], keyword()) ::
  {:ok, [map()]} | {:error, Exception.t()}

Returns metadata for a list of paths. See HuggingfaceClient.paths_info/3.