Vllm.Config.ECTransferConfig (VLLM v0.3.0)

Copy Markdown View Source

Configuration for distributed EC cache transfer.

Summary

Types

t()

Functions

compute_hash(ref, opts \\ [])

WARNING: Whenever a new field is added to this config,

ec_buffer_device(ref)
ec_buffer_size(ref)
ec_connector(ref)
ec_connector_module_path(ref)
ec_ip(ref)
ec_parallel_size(ref)
ec_port(ref)
ec_rank(ref)
ec_role(ref)
engine_id(ref)
get_from_extra_config(ref, key, default, opts \\ [])

Python method ECTransferConfig.get_from_extra_config.

is_ec_consumer(ref)
is_ec_producer(ref)
is_ec_transfer_instance(ref)
new(dataclass_self__, args, kwargs, opts \\ [])

Constructs ECTransferConfig.

Types

t()

@opaque t()

Functions

compute_hash(ref, opts \\ [])

@spec compute_hash(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

WARNING: Whenever a new field is added to this config,

ensure that it is included in the factors list if it affects the computation graph.

Provide a hash that uniquely identifies all the configs that affect the structure of the computation graph from input ids/embeddings to the final hidden states, excluding anything before input ids/embeddings and after the final hidden states.

Returns

  • String.t()

ec_buffer_device(ref)

@spec ec_buffer_device(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

ec_buffer_size(ref)

@spec ec_buffer_size(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

ec_connector(ref)

@spec ec_connector(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

ec_connector_module_path(ref)

@spec ec_connector_module_path(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

ec_ip(ref)

@spec ec_ip(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

ec_parallel_size(ref)

@spec ec_parallel_size(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

ec_port(ref)

@spec ec_port(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

ec_rank(ref)

@spec ec_rank(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

ec_role(ref)

@spec ec_role(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

engine_id(ref)

@spec engine_id(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

get_from_extra_config(ref, key, default, opts \\ [])

@spec get_from_extra_config(SnakeBridge.Ref.t(), term(), term(), keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

Python method ECTransferConfig.get_from_extra_config.

Parameters

  • key (term())
  • default (term())

Returns

  • term()

is_ec_consumer(ref)

@spec is_ec_consumer(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

is_ec_producer(ref)

@spec is_ec_producer(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

is_ec_transfer_instance(ref)

@spec is_ec_transfer_instance(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

new(dataclass_self__, args, kwargs, opts \\ [])

@spec new(term(), term(), term(), keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Constructs ECTransferConfig.

Parameters

  • dataclass_self__ (term())
  • args (term())
  • kwargs (term())