Vllm.Config.ProfilerConfig (VLLM v0.3.0)

Copy Markdown View Source

Dataclass which contains profiler config for the engine.

Summary

Types

t()

Functions

_get_from_env_if_set(ref, field_name, env_var_name, opts \\ [])

Get field from env var if set, with deprecation warning.

_set_from_env_if_set(ref, field_name, env_var_name, args, opts \\ [])

Set field from env var if set, with deprecation warning.

_validate_profiler_config(ref, opts \\ [])

Python method ProfilerConfig._validate_profiler_config.

compute_hash(ref, opts \\ [])

WARNING: Whenever a new field is added to this config,

delay_iterations(ref)
ignore_frontend(ref)
max_iterations(ref)
new(dataclass_self__, args, kwargs, opts \\ [])

Constructs ProfilerConfig.

profiler(ref)
torch_profiler_dir(ref)
torch_profiler_dump_cuda_time_total(ref)
torch_profiler_record_shapes(ref)
torch_profiler_use_gzip(ref)
torch_profiler_with_flops(ref)
torch_profiler_with_memory(ref)
torch_profiler_with_stack(ref)

Types

t()

@opaque t()

Functions

_get_from_env_if_set(ref, field_name, env_var_name, opts \\ [])

@spec _get_from_env_if_set(SnakeBridge.Ref.t(), String.t(), String.t(), keyword()) ::
  {:ok, nil} | {:error, Snakepit.Error.t()}

Get field from env var if set, with deprecation warning.

Parameters

  • field_name (String.t())
  • env_var_name (String.t())

Returns

  • nil

_set_from_env_if_set(ref, field_name, env_var_name, args, opts \\ [])

@spec _set_from_env_if_set(
  SnakeBridge.Ref.t(),
  String.t(),
  String.t(),
  [term()],
  keyword()
) ::
  {:ok, nil} | {:error, Snakepit.Error.t()}

Set field from env var if set, with deprecation warning.

Parameters

  • field_name (String.t())
  • env_var_name (String.t())
  • to_bool (boolean() default: True)
  • to_int (boolean() default: False)

Returns

  • nil

_validate_profiler_config(ref, opts \\ [])

@spec _validate_profiler_config(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, term()} | {:error, Snakepit.Error.t()}

Python method ProfilerConfig._validate_profiler_config.

Returns

  • term()

compute_hash(ref, opts \\ [])

@spec compute_hash(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

WARNING: Whenever a new field is added to this config,

ensure that it is included in the factors list if it affects the computation graph.

Provide a hash that uniquely identifies all the configs that affect the structure of the computation graph from input ids/embeddings to the final hidden states, excluding anything before input ids/embeddings and after the final hidden states.

Returns

  • String.t()

delay_iterations(ref)

@spec delay_iterations(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

ignore_frontend(ref)

@spec ignore_frontend(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

max_iterations(ref)

@spec max_iterations(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

new(dataclass_self__, args, kwargs, opts \\ [])

@spec new(term(), term(), term(), keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Constructs ProfilerConfig.

Parameters

  • dataclass_self__ (term())
  • args (term())
  • kwargs (term())

profiler(ref)

@spec profiler(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

torch_profiler_dir(ref)

@spec torch_profiler_dir(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

torch_profiler_dump_cuda_time_total(ref)

@spec torch_profiler_dump_cuda_time_total(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

torch_profiler_record_shapes(ref)

@spec torch_profiler_record_shapes(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

torch_profiler_use_gzip(ref)

@spec torch_profiler_use_gzip(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

torch_profiler_with_flops(ref)

@spec torch_profiler_with_flops(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

torch_profiler_with_memory(ref)

@spec torch_profiler_with_memory(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

torch_profiler_with_stack(ref)

@spec torch_profiler_with_stack(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}