HfHub.Cache (HfHub v0.1.3)

View Source

Cache management for downloaded files.

Provides functions to manage the local cache of downloaded HuggingFace files, including checking cache status, retrieving paths, and evicting old files.

Examples

# Check if a file is cached
cached? = HfHub.Cache.cached?(
  repo_id: "bert-base-uncased",
  filename: "config.json"
)

# Get cache path for a file
{:ok, path} = HfHub.Cache.cache_path(
  repo_id: "bert-base-uncased",
  filename: "config.json"
)

# Clear cache for a repository
:ok = HfHub.Cache.clear_cache(repo_id: "bert-base-uncased")

# Get cache statistics
{:ok, stats} = HfHub.Cache.cache_stats()

Summary

Types

cache_stats()

Functions

cache_path(opts)

Gets the local cache path for a file.

cache_stats()

Gets cache statistics.

cached?(opts)

Checks if a file is cached locally.

clear_cache(opts \\ [])

Clears cached files.

evict_lru(opts)

Evicts least recently used files to free up space.

validate_integrity()

Validates cache integrity.

Types

cache_stats()

@type cache_stats() :: %{
  total_size: non_neg_integer(),
  file_count: non_neg_integer(),
  repos: [String.t()],
  last_accessed: DateTime.t() | nil
}

Functions

cache_path(opts)

@spec cache_path(keyword()) :: {:ok, Path.t()} | {:error, :not_cached}

Gets the local cache path for a file.

Returns {:ok, path} if the file is cached, {:error, :not_cached} otherwise.

Options

  • :repo_id - Repository ID
  • :filename - Name of the file
  • :repo_type - Type of repository. Defaults to :model.
  • :revision - Git revision. Defaults to "main".

Examples

{:ok, path} = HfHub.Cache.cache_path(
  repo_id: "bert-base-uncased",
  filename: "config.json"
)

cache_stats()

@spec cache_stats() :: {:ok, cache_stats()} | {:error, term()}

Gets cache statistics.

Returns information about cache size, file count, and repositories.

Examples

{:ok, stats} = HfHub.Cache.cache_stats()
IO.inspect(stats.total_size)   # Total bytes
IO.inspect(stats.file_count)   # Number of files
IO.inspect(stats.repos)        # List of cached repos

cached?(opts)

@spec cached?(keyword()) :: boolean()

Checks if a file is cached locally.

Options

  • :repo_id - Repository ID
  • :filename - Name of the file
  • :repo_type - Type of repository. Defaults to :model.
  • :revision - Git revision. Defaults to "main".

Examples

cached? = HfHub.Cache.cached?(
  repo_id: "bert-base-uncased",
  filename: "config.json"
)

clear_cache(opts \\ [])

@spec clear_cache(keyword()) :: :ok | {:error, term()}

Clears cached files.

Options

  • :repo_id - Clear cache for specific repository. If not provided, clears all.
  • :repo_type - Type of repository. Defaults to :model.

Examples

# Clear cache for specific repo
:ok = HfHub.Cache.clear_cache(repo_id: "bert-base-uncased")

# Clear all cache
:ok = HfHub.Cache.clear_cache()

evict_lru(opts)

@spec evict_lru(keyword()) :: :ok | {:error, term()}

Evicts least recently used files to free up space.

Options

  • :target_size - Target cache size in bytes
  • :max_age - Maximum age of files in seconds

Examples

# Evict files to get under 5GB
:ok = HfHub.Cache.evict_lru(target_size: 5 * 1024 * 1024 * 1024)

# Evict files older than 30 days
:ok = HfHub.Cache.evict_lru(max_age: 30 * 24 * 60 * 60)

validate_integrity()

@spec validate_integrity() :: {:ok, map()} | {:error, term()}

Validates cache integrity.

Checks that all cached files have valid checksums and removes corrupted files.

Examples

{:ok, report} = HfHub.Cache.validate_integrity()