Vllm.Config.DeviceConfig (VLLM v0.3.0)

Copy Markdown View Source

Configuration for the device to use for vLLM execution.

Summary

Functions

WARNING: Whenever a new field is added to this config,

Types

t()

@opaque t()

Functions

compute_hash(ref, opts \\ [])

@spec compute_hash(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

WARNING: Whenever a new field is added to this config,

ensure that it is included in the factors list if it affects the computation graph.

Provide a hash that uniquely identifies all the configs that affect the structure of the computation graph from input ids/embeddings to the final hidden states, excluding anything before input ids/embeddings and after the final hidden states.

Returns

  • String.t()

device(ref)

@spec device(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

new(dataclass_self__, args, kwargs, opts \\ [])

@spec new(term(), term(), term(), keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Constructs DeviceConfig.

Parameters

  • dataclass_self__ (term())
  • args (term())
  • kwargs (term())