Vllm.Config.EPLBConfig (VLLM v0.3.0)

Configuration for Expert Parallel Load Balancing (EP).

Summary

t()

Python method EPLBConfig._validate_eplb_config.

Constructs EPLBConfig.

policy(ref)

use_async(ref)

window_size(ref)

@opaque t()

@spec _validate_eplb_config(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, term()} | {:error, Snakepit.Error.t()}

Python method EPLBConfig._validate_eplb_config.

@spec log_balancedness(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

@spec log_balancedness_interval(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

@spec new(term(), term(), term(), keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Constructs EPLBConfig.

@spec num_redundant_experts(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

@spec policy(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

@spec step_interval(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

@spec use_async(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

@spec window_size(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}