Vllm.Config.ObservabilityConfig (VLLM v0.3.0)

Copy Markdown View Source

Configuration for observability - metrics and tracing.

Summary

Types

t()

Functions

_validate_collect_detailed_traces(ref, value, opts \\ [])

Handle the legacy case where users might provide a comma-separated

_validate_otlp_traces_endpoint(ref, value, opts \\ [])

Python method ObservabilityConfig._validate_otlp_traces_endpoint.

_validate_show_hidden_metrics_for_version(ref, value, opts \\ [])

Python method ObservabilityConfig._validate_show_hidden_metrics_for_version.

_validate_tracing_config(ref, opts \\ [])

Python method ObservabilityConfig._validate_tracing_config.

collect_detailed_traces(ref)
collect_model_execute_time(ref)
collect_model_forward_time(ref)
compute_hash(ref, opts \\ [])

WARNING: Whenever a new field is added to this config,

cudagraph_metrics(ref)
enable_layerwise_nvtx_tracing(ref)
enable_logging_iteration_details(ref)
enable_mfu_metrics(ref)
enable_mm_processor_stats(ref)
kv_cache_metrics(ref)
kv_cache_metrics_sample(ref)
new(dataclass_self__, args, kwargs, opts \\ [])

Constructs ObservabilityConfig.

otlp_traces_endpoint(ref)
show_hidden_metrics(ref)
show_hidden_metrics_for_version(ref)

Types

t()

@opaque t()

Functions

_validate_collect_detailed_traces(ref, value, opts \\ [])

@spec _validate_collect_detailed_traces(SnakeBridge.Ref.t(), term(), keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

Handle the legacy case where users might provide a comma-separated

string instead of a list of strings.

Parameters

  • value (term())

Returns

  • term()

_validate_otlp_traces_endpoint(ref, value, opts \\ [])

@spec _validate_otlp_traces_endpoint(SnakeBridge.Ref.t(), term(), keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

Python method ObservabilityConfig._validate_otlp_traces_endpoint.

Parameters

  • value (term())

Returns

  • term()

_validate_show_hidden_metrics_for_version(ref, value, opts \\ [])

@spec _validate_show_hidden_metrics_for_version(
  SnakeBridge.Ref.t(),
  term(),
  keyword()
) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

Python method ObservabilityConfig._validate_show_hidden_metrics_for_version.

Parameters

  • value (term())

Returns

  • term()

_validate_tracing_config(ref, opts \\ [])

@spec _validate_tracing_config(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, term()} | {:error, Snakepit.Error.t()}

Python method ObservabilityConfig._validate_tracing_config.

Returns

  • term()

collect_detailed_traces(ref)

@spec collect_detailed_traces(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

collect_model_execute_time(ref)

@spec collect_model_execute_time(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

collect_model_forward_time(ref)

@spec collect_model_forward_time(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

compute_hash(ref, opts \\ [])

@spec compute_hash(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

WARNING: Whenever a new field is added to this config,

ensure that it is included in the factors list if it affects the computation graph.

Provide a hash that uniquely identifies all the configs that affect the structure of the computation graph from input ids/embeddings to the final hidden states, excluding anything before input ids/embeddings and after the final hidden states.

Returns

  • String.t()

cudagraph_metrics(ref)

@spec cudagraph_metrics(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

enable_layerwise_nvtx_tracing(ref)

@spec enable_layerwise_nvtx_tracing(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

enable_logging_iteration_details(ref)

@spec enable_logging_iteration_details(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

enable_mfu_metrics(ref)

@spec enable_mfu_metrics(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

enable_mm_processor_stats(ref)

@spec enable_mm_processor_stats(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

kv_cache_metrics(ref)

@spec kv_cache_metrics(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

kv_cache_metrics_sample(ref)

@spec kv_cache_metrics_sample(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

new(dataclass_self__, args, kwargs, opts \\ [])

@spec new(term(), term(), term(), keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Constructs ObservabilityConfig.

Parameters

  • dataclass_self__ (term())
  • args (term())
  • kwargs (term())

otlp_traces_endpoint(ref)

@spec otlp_traces_endpoint(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

show_hidden_metrics(ref)

@spec show_hidden_metrics(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

show_hidden_metrics_for_version(ref)

@spec show_hidden_metrics_for_version(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}