Vllm.Config.LoadConfig (VLLM v0.3.0)

Copy Markdown View Source

Configuration for loading the model weights.

Summary

Types

t()

Functions

_lowercase_load_format(ref, load_format, opts \\ [])

Python method LoadConfig._lowercase_load_format.

_validate_ignore_patterns(ref, ignore_patterns, opts \\ [])

Python method LoadConfig._validate_ignore_patterns.

compute_hash(ref, opts \\ [])

WARNING: Whenever a new field is added to this config,

device(ref)
download_dir(ref)
ignore_patterns(ref)
load_format(ref)
model_loader_extra_config(ref)
new(dataclass_self__, args, kwargs, opts \\ [])

Constructs LoadConfig.

pt_load_map_location(ref)
safetensors_load_strategy(ref)
use_tqdm_on_load(ref)

Types

t()

@opaque t()

Functions

_lowercase_load_format(ref, load_format, opts \\ [])

@spec _lowercase_load_format(SnakeBridge.Ref.t(), String.t(), keyword()) ::
  {:ok, String.t()} | {:error, Snakepit.Error.t()}

Python method LoadConfig._lowercase_load_format.

Parameters

  • load_format (String.t())

Returns

  • String.t()

_validate_ignore_patterns(ref, ignore_patterns, opts \\ [])

@spec _validate_ignore_patterns(SnakeBridge.Ref.t(), term(), keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

Python method LoadConfig._validate_ignore_patterns.

Parameters

  • ignore_patterns (term())

Returns

  • term()

compute_hash(ref, opts \\ [])

@spec compute_hash(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

WARNING: Whenever a new field is added to this config,

ensure that it is included in the factors list if it affects the computation graph.

Provide a hash that uniquely identifies all the configs that affect the structure of the computation graph from input ids/embeddings to the final hidden states, excluding anything before input ids/embeddings and after the final hidden states.

Returns

  • String.t()

device(ref)

@spec device(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

download_dir(ref)

@spec download_dir(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

ignore_patterns(ref)

@spec ignore_patterns(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

load_format(ref)

@spec load_format(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

model_loader_extra_config(ref)

@spec model_loader_extra_config(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

new(dataclass_self__, args, kwargs, opts \\ [])

@spec new(term(), term(), term(), keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Constructs LoadConfig.

Parameters

  • dataclass_self__ (term())
  • args (term())
  • kwargs (term())

pt_load_map_location(ref)

@spec pt_load_map_location(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

safetensors_load_strategy(ref)

@spec safetensors_load_strategy(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

use_tqdm_on_load(ref)

@spec use_tqdm_on_load(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}