Tinkex.CheckpointDownload (Tinkex v0.3.4)
View SourceDownload and extract checkpoint archives with streaming support.
Provides memory-efficient checkpoint downloads using Finch.stream_while/5.
Downloads are streamed directly to disk with O(1) memory usage, making it
safe to download large checkpoint files (100MB-GBs) without risk of OOM errors.
Features
- Streaming downloads - O(1) memory usage regardless of file size
- Progress callbacks - Track download progress in real-time
- Automatic extraction - Downloads and extracts tar archives in one operation
- Force overwrite - Optional overwrite of existing checkpoint directories
Examples
# Basic download with automatic extraction
{:ok, service_pid} = Tinkex.ServiceClient.start_link(config: config)
{:ok, rest_client} = Tinkex.ServiceClient.create_rest_client(service_pid)
{:ok, result} = Tinkex.CheckpointDownload.download(
rest_client,
"tinker://run-123/weights/0001",
output_dir: "./models",
force: true
)
IO.puts("Downloaded to: #{result.destination}")
# Download with progress tracking
progress_fn = fn downloaded, total ->
percent = if total > 0, do: Float.round(downloaded / total * 100, 1), else: 0
IO.write("\rProgress: #{percent}% (#{downloaded} / #{total} bytes)")
end
{:ok, result} = Tinkex.CheckpointDownload.download(
rest_client,
"tinker://run-123/weights/0001",
output_dir: "./models",
progress: progress_fn
)
Summary
Functions
Download and extract a checkpoint.
Functions
@spec download(Tinkex.RestClient.t(), String.t(), keyword()) :: {:ok, map()} | {:error, term()}
Download and extract a checkpoint.
Options
:output_dir- Parent directory for extraction (default: current directory):force- Overwrite existing directory (default: false):progress- Progress callback functionfn(downloaded, total) -> any
Returns
{:ok, %{destination: path, checkpoint_path: path}}on success{:error, {:exists, path}}if target exists and force is false{:error, {:invalid_path, message}}if checkpoint path is invalid{:error, reason}for other failures