Vllm.Plugins (VLLM v0.3.0)

Copy Markdown View Source

Submodule bindings for vllm.plugins.

Version

  • Requested: 0.14.0
  • Observed at generation: 0.14.0

Runtime Options

All functions accept a __runtime__ option for controlling execution behavior:

Vllm.Plugins.some_function(args, __runtime__: [timeout: 120_000])

Supported runtime options

  • :timeout - Call timeout in milliseconds (default: 120,000ms / 2 minutes)
  • :timeout_profile - Use a named profile (:default, :ml_inference, :batch_job, :streaming)
  • :stream_timeout - Timeout for streaming operations (default: 1,800,000ms / 30 minutes)
  • :session_id - Override the session ID for this call
  • :pool_name - Target a specific Snakepit pool (multi-pool setups)
  • :affinity - Override session affinity (:hint, :strict_queue, :strict_fail_fast)

Timeout Profiles

  • :default - 2 minute timeout for regular calls
  • :ml_inference - 10 minute timeout for ML/LLM workloads
  • :batch_job - Unlimited timeout for long-running jobs
  • :streaming - 2 minute timeout, 30 minute stream_timeout

Example with timeout override

# For a long-running ML inference call
Vllm.Plugins.predict(data, __runtime__: [timeout_profile: :ml_inference])

# Or explicit timeout
Vllm.Plugins.predict(data, __runtime__: [timeout: 600_000])

# Route to a pool and enforce strict affinity
Vllm.Plugins.predict(data, __runtime__: [pool_name: :strict_pool, affinity: :strict_queue])

See SnakeBridge.Defaults for global timeout configuration.

Summary

Functions

default_plugins_group()

Python module attribute vllm.plugins.DEFAULT_PLUGINS_GROUP.

io_processor_plugins_group()

Python module attribute vllm.plugins.IO_PROCESSOR_PLUGINS_GROUP.

load_general_plugins(opts \\ [])

WARNING: plugins can be loaded for multiple times in different

load_plugins_by_group(group, opts \\ [])

Python binding for vllm.plugins.load_plugins_by_group.

logger()

Python module attribute vllm.plugins.logger.

platform_plugins_group()

Python module attribute vllm.plugins.PLATFORM_PLUGINS_GROUP.

plugins_loaded()

Python module attribute vllm.plugins.plugins_loaded.

stat_logger_plugins_group()

Python module attribute vllm.plugins.STAT_LOGGER_PLUGINS_GROUP.

Functions

default_plugins_group()

@spec default_plugins_group() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

Python module attribute vllm.plugins.DEFAULT_PLUGINS_GROUP.

Returns

  • String.t()

io_processor_plugins_group()

@spec io_processor_plugins_group() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

Python module attribute vllm.plugins.IO_PROCESSOR_PLUGINS_GROUP.

Returns

  • String.t()

load_general_plugins(opts \\ [])

@spec load_general_plugins(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

WARNING: plugins can be loaded for multiple times in different

processes. They should be designed in a way that they can be loaded multiple times without causing issues.

Returns

  • term()

load_plugins_by_group(group, opts \\ [])

@spec load_plugins_by_group(
  String.t(),
  keyword()
) :: {:ok, %{optional(String.t()) => term()}} | {:error, Snakepit.Error.t()}

Python binding for vllm.plugins.load_plugins_by_group.

Parameters

  • group (String.t())

Returns

  • %{optional(String.t()) => term()}

logger()

@spec logger() :: {:ok, term()} | {:error, Snakepit.Error.t()}

Python module attribute vllm.plugins.logger.

Returns

  • term()

platform_plugins_group()

@spec platform_plugins_group() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

Python module attribute vllm.plugins.PLATFORM_PLUGINS_GROUP.

Returns

  • String.t()

plugins_loaded()

@spec plugins_loaded() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}

Python module attribute vllm.plugins.plugins_loaded.

Returns

  • boolean()

stat_logger_plugins_group()

@spec stat_logger_plugins_group() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}

Python module attribute vllm.plugins.STAT_LOGGER_PLUGINS_GROUP.

Returns

  • String.t()