Submodule bindings for vllm.envs.
Version
- Requested: 0.14.0
- Observed at generation: 0.14.0
Runtime Options
All functions accept a __runtime__ option for controlling execution behavior:
Vllm.Envs.some_function(args, __runtime__: [timeout: 120_000])Supported runtime options
:timeout- Call timeout in milliseconds (default: 120,000ms / 2 minutes):timeout_profile- Use a named profile (:default,:ml_inference,:batch_job,:streaming):stream_timeout- Timeout for streaming operations (default: 1,800,000ms / 30 minutes):session_id- Override the session ID for this call:pool_name- Target a specific Snakepit pool (multi-pool setups):affinity- Override session affinity (:hint,:strict_queue,:strict_fail_fast)
Timeout Profiles
:default- 2 minute timeout for regular calls:ml_inference- 10 minute timeout for ML/LLM workloads:batch_job- Unlimited timeout for long-running jobs:streaming- 2 minute timeout, 30 minute stream_timeout
Example with timeout override
# For a long-running ML inference call
Vllm.Envs.predict(data, __runtime__: [timeout_profile: :ml_inference])
# Or explicit timeout
Vllm.Envs.predict(data, __runtime__: [timeout: 600_000])
# Route to a pool and enforce strict affinity
Vllm.Envs.predict(data, __runtime__: [pool_name: :strict_pool, affinity: :strict_queue])See SnakeBridge.Defaults for global timeout configuration.
Summary
Functions
Python binding for vllm.envs.__dir__.
Gets environment variables lazily.
Checked if getattr is wrapped with functools.cache
Python binding for vllm.envs.CMAKE_BUILD_TYPE.
Return env vars used for torch.compile cache keys.
Python binding for vllm.envs.CUDA_VISIBLE_DEVICES.
Python binding for vllm.envs.disable_compile_cache.
Resets the environment variables cache. It could be used to isolate environments
Enables caching of environment variables. This is useful for performance
Create a lambda that validates environment variable
Creates a lambda which that validates environment variable
Create a lambda that validates environment variable against allowed choices
Python module attribute vllm.envs.environment_variables.
Python binding for vllm.envs.get_default_cache_root.
Python binding for vllm.envs.get_default_config_root.
Python binding for vllm.envs.get_env_or_set_default.
Get the port from VLLM_PORT environment variable.
Check if an environment variable is explicitly set.
Python module attribute vllm.envs.K_SCALE_CONSTANT.
Python binding for vllm.envs.LD_LIBRARY_PATH.
Python module attribute vllm.envs.LOCAL_RANK.
Python module attribute vllm.envs.logger.
Python binding for vllm.envs.MAX_JOBS.
Python binding for vllm.envs.maybe_convert_bool.
Python binding for vllm.envs.maybe_convert_int.
Python module attribute vllm.envs.NO_COLOR.
Python binding for vllm.envs.NVCC_THREADS.
Python module attribute vllm.envs.Q_SCALE_CONSTANT.
Python binding for vllm.envs.S3_ACCESS_KEY_ID.
Python binding for vllm.envs.S3_ENDPOINT_URL.
Python binding for vllm.envs.S3_SECRET_ACCESS_KEY.
Python binding for vllm.envs.use_aot_compile.
Python module attribute vllm.envs.V_SCALE_CONSTANT.
Python module attribute vllm.envs.VERBOSE.
Python binding for vllm.envs.VLLM_ALL2ALL_BACKEND.
Python module attribute vllm.envs.VLLM_ALLOW_CHUNKED_LOCAL_ATTN_WITH_HYBRID_KV_CACHE.
Python module attribute vllm.envs.VLLM_ALLOW_INSECURE_SERIALIZATION.
Python module attribute vllm.envs.VLLM_ALLOW_LONG_MAX_MODEL_LEN.
Python module attribute vllm.envs.VLLM_ALLOW_RUNTIME_LORA_UPDATING.
Python module attribute vllm.envs.VLLM_ALLREDUCE_USE_SYMM_MEM.
Python binding for vllm.envs.VLLM_API_KEY.
Python module attribute vllm.envs.VLLM_ASSETS_CACHE.
Python module attribute vllm.envs.VLLM_ASSETS_CACHE_MODEL_CLEAN.
Python module attribute vllm.envs.VLLM_AUDIO_FETCH_TIMEOUT.
Python module attribute vllm.envs.VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER.
Python module attribute vllm.envs.VLLM_CACHE_ROOT.
Python module attribute vllm.envs.VLLM_COMPILE_CACHE_SAVE_FORMAT.
Python module attribute vllm.envs.VLLM_COMPUTE_NANS_IN_LOGITS.
Python module attribute vllm.envs.VLLM_CONFIG_ROOT.
Python module attribute vllm.envs.VLLM_CONFIGURE_LOGGING.
Python binding for vllm.envs.VLLM_CPU_KVCACHE_SPACE.
Python binding for vllm.envs.VLLM_CPU_NUM_OF_RESERVED_CPU.
Python module attribute vllm.envs.VLLM_CPU_OMP_THREADS_BIND.
Python module attribute vllm.envs.VLLM_CPU_SGL_KERNEL.
Python binding for vllm.envs.VLLM_CUDART_SO_PATH.
Python module attribute vllm.envs.VLLM_CUSTOM_SCOPES_FOR_PROFILING.
Python module attribute vllm.envs.VLLM_DBO_COMM_SMS.
Python binding for vllm.envs.VLLM_DEBUG_DUMP_PATH.
Python module attribute vllm.envs.VLLM_DEBUG_LOG_API_SERVER_RESPONSE.
Python module attribute vllm.envs.VLLM_DEBUG_MFU_METRICS.
Python module attribute vllm.envs.VLLM_DEBUG_WORKSPACE.
Python module attribute vllm.envs.VLLM_DEEP_GEMM_WARMUP.
Python module attribute vllm.envs.VLLM_DEEPEP_BUFFER_SIZE_MB.
Python module attribute vllm.envs.VLLM_DEEPEP_HIGH_THROUGHPUT_FORCE_INTRA_NODE.
Python module attribute vllm.envs.VLLM_DEEPEP_LOW_LATENCY_USE_MNNVL.
Python module attribute vllm.envs.VLLM_DEEPEPLL_NVFP4_DISPATCH.
Python module attribute vllm.envs.VLLM_DISABLE_COMPILE_CACHE.
Python binding for vllm.envs.VLLM_DISABLE_LOG_LOGO.
Python module attribute vllm.envs.VLLM_DISABLE_PYNCCL.
Python module attribute vllm.envs.VLLM_DISABLE_SHARED_EXPERTS_STREAM.
Python module attribute vllm.envs.VLLM_DISABLED_KERNELS.
Python module attribute vllm.envs.VLLM_DO_NOT_TRACK.
Python module attribute vllm.envs.VLLM_DOCKER_BUILD_CONTEXT.
Python module attribute vllm.envs.VLLM_DP_MASTER_IP.
Python module attribute vllm.envs.VLLM_DP_MASTER_PORT.
Python module attribute vllm.envs.VLLM_DP_RANK.
Python module attribute vllm.envs.VLLM_DP_RANK_LOCAL.
Python module attribute vllm.envs.VLLM_DP_SIZE.
Python module attribute vllm.envs.VLLM_ENABLE_CUDAGRAPH_GC.
Python module attribute vllm.envs.VLLM_ENABLE_FUSED_MOE_ACTIVATION_CHUNKING.
Python module attribute vllm.envs.VLLM_ENABLE_INDUCTOR_COORDINATE_DESCENT_TUNING.
Python module attribute vllm.envs.VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE.
Python module attribute vllm.envs.VLLM_ENABLE_MOE_DP_CHUNK.
Python module attribute vllm.envs.VLLM_ENABLE_RESPONSES_API_STORE.
Python module attribute vllm.envs.VLLM_ENABLE_V1_MULTIPROCESSING.
Python module attribute vllm.envs.VLLM_ENGINE_ITERATION_TIMEOUT_S.
Python module attribute vllm.envs.VLLM_ENGINE_READY_TIMEOUT_S.
Python module attribute vllm.envs.VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS.
Python module attribute vllm.envs.VLLM_FLASHINFER_MOE_BACKEND.
Python module attribute vllm.envs.VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE.
Python module attribute vllm.envs.VLLM_FLOAT32_MATMUL_PRECISION.
Python module attribute vllm.envs.VLLM_FORCE_AOT_LOAD.
Python module attribute vllm.envs.VLLM_FUSED_MOE_CHUNK_SIZE.
Python module attribute vllm.envs.VLLM_GC_DEBUG.
Python module attribute vllm.envs.VLLM_GPT_OSS_HARMONY_SYSTEM_INSTRUCTIONS.
Python module attribute vllm.envs.VLLM_GPT_OSS_SYSTEM_TOOL_MCP_LABELS.
Python module attribute vllm.envs.VLLM_HAS_FLASHINFER_CUBIN.
Python module attribute vllm.envs.VLLM_HOST_IP.
Python module attribute vllm.envs.VLLM_HTTP_TIMEOUT_KEEP_ALIVE.
Python module attribute vllm.envs.VLLM_IMAGE_FETCH_TIMEOUT.
Python module attribute vllm.envs.VLLM_KEEP_ALIVE_ON_ENGINE_DEATH.
Python binding for vllm.envs.VLLM_KV_CACHE_LAYOUT.
Python module attribute vllm.envs.VLLM_KV_EVENTS_USE_INT_BLOCK_HASHES.
Python module attribute vllm.envs.VLLM_LOG_BATCHSIZE_INTERVAL.
Python module attribute vllm.envs.VLLM_LOG_MODEL_INSPECTION.
Python module attribute vllm.envs.VLLM_LOG_STATS_INTERVAL.
Python module attribute vllm.envs.VLLM_LOGGING_COLOR.
Python binding for vllm.envs.VLLM_LOGGING_CONFIG_PATH.
Python module attribute vllm.envs.VLLM_LOGGING_LEVEL.
Python module attribute vllm.envs.VLLM_LOGGING_PREFIX.
Python module attribute vllm.envs.VLLM_LOGGING_STREAM.
Python module attribute vllm.envs.VLLM_LOOPBACK_IP.
Python binding for vllm.envs.VLLM_LORA_RESOLVER_CACHE_DIR.
Python module attribute vllm.envs.VLLM_MAIN_CUDA_VERSION.
Python binding for vllm.envs.VLLM_MARLIN_INPUT_DTYPE.
Python module attribute vllm.envs.VLLM_MARLIN_USE_ATOMIC_ADD.
Python module attribute vllm.envs.VLLM_MAX_AUDIO_CLIP_FILESIZE_MB.
Python module attribute vllm.envs.VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE.
Python module attribute vllm.envs.VLLM_MEDIA_CONNECTOR.
Python module attribute vllm.envs.VLLM_MEDIA_LOADING_THREAD_COUNT.
Python module attribute vllm.envs.VLLM_MEDIA_URL_ALLOW_REDIRECTS.
Python module attribute vllm.envs.VLLM_MLA_DISABLE.
Python binding for vllm.envs.VLLM_MM_HASHER_ALGORITHM.
Python binding for vllm.envs.VLLM_MODEL_REDIRECT_PATH.
Python module attribute vllm.envs.VLLM_MOE_DP_CHUNK_SIZE.
Python module attribute vllm.envs.VLLM_MOE_USE_DEEP_GEMM.
Python module attribute vllm.envs.VLLM_MOONCAKE_ABORT_REQUEST_TIMEOUT.
Python module attribute vllm.envs.VLLM_MOONCAKE_BOOTSTRAP_PORT.
Python module attribute vllm.envs.VLLM_MORIIO_CONNECTOR_READ_MODE.
Python module attribute vllm.envs.VLLM_MORIIO_NUM_WORKERS.
Python module attribute vllm.envs.VLLM_MORIIO_POST_BATCH_SIZE.
Python module attribute vllm.envs.VLLM_MORIIO_QP_PER_TRANSFER.
Python module attribute vllm.envs.VLLM_MQ_MAX_CHUNK_BYTES_MB.
Python module attribute vllm.envs.VLLM_MSGPACK_ZERO_COPY_THRESHOLD.
Python binding for vllm.envs.VLLM_MXFP4_USE_MARLIN.
Python binding for vllm.envs.VLLM_NCCL_INCLUDE_PATH.
Python binding for vllm.envs.VLLM_NCCL_SO_PATH.
Python module attribute vllm.envs.VLLM_NIXL_ABORT_REQUEST_TIMEOUT.
Python module attribute vllm.envs.VLLM_NIXL_SIDE_CHANNEL_HOST.
Python module attribute vllm.envs.VLLM_NIXL_SIDE_CHANNEL_PORT.
Python module attribute vllm.envs.VLLM_NO_USAGE_STATS.
Python binding for vllm.envs.VLLM_NVFP4_GEMM_BACKEND.
Python module attribute vllm.envs.VLLM_NVTX_SCOPES_FOR_PROFILING.
Python module attribute vllm.envs.VLLM_OBJECT_STORAGE_SHM_BUFFER_NAME.
Python binding for vllm.envs.VLLM_PATTERN_MATCH_DEBUG.
Python binding for vllm.envs.VLLM_PLUGINS.
Python binding for vllm.envs.VLLM_PORT.
Python binding for vllm.envs.VLLM_PP_LAYER_PARTITION.
Python binding for vllm.envs.VLLM_PROFILER_DELAY_ITERS.
Python binding for vllm.envs.VLLM_PROFILER_MAX_ITERS.
Python module attribute vllm.envs.VLLM_RANDOMIZE_DP_DUMMY_INPUTS.
Python module attribute vllm.envs.VLLM_RAY_BUNDLE_INDICES.
Python module attribute vllm.envs.VLLM_RAY_DP_PACK_STRATEGY.
Python module attribute vllm.envs.VLLM_RAY_PER_WORKER_GPUS.
Python module attribute vllm.envs.VLLM_RINGBUFFER_WARNING_INTERVAL.
Python module attribute vllm.envs.VLLM_ROCM_CUSTOM_PAGED_ATTN.
Python module attribute vllm.envs.VLLM_ROCM_FP8_MFMA_PAGE_ATTN.
Python module attribute vllm.envs.VLLM_ROCM_FP8_PADDING.
Python module attribute vllm.envs.VLLM_ROCM_MOE_PADDING.
Python module attribute vllm.envs.VLLM_ROCM_QUICK_REDUCE_CAST_BF16_TO_FP16.
Python binding for vllm.envs.VLLM_ROCM_QUICK_REDUCE_MAX_SIZE_BYTES_MB.
Python module attribute vllm.envs.VLLM_ROCM_QUICK_REDUCE_QUANTIZATION.
Python binding for vllm.envs.VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT.
Python module attribute vllm.envs.VLLM_ROCM_SLEEP_MEM_CHUNK_SIZE.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_FP4_ASM_GEMM.
Python binding for vllm.envs.VLLM_ROCM_USE_AITER_FP4BMM.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_FP8BMM.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_LINEAR.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_MHA.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_MLA.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_MOE.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_PAGED_ATTN.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_RMSNORM.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_TRITON_GEMM.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_TRITON_ROPE.
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION.
Python module attribute vllm.envs.VLLM_ROCM_USE_SKINNY_GEMM.
Python module attribute vllm.envs.VLLM_RPC_BASE_PATH.
Python module attribute vllm.envs.VLLM_RPC_TIMEOUT.
Python module attribute vllm.envs.VLLM_SERVER_DEV_MODE.
Python module attribute vllm.envs.VLLM_SHARED_EXPERTS_STREAM_TOKEN_THRESHOLD.
Python module attribute vllm.envs.VLLM_SKIP_P2P_CHECK.
Python module attribute vllm.envs.VLLM_SKIP_PRECOMPILED_VERSION_SUFFIX.
Python module attribute vllm.envs.VLLM_SLEEP_WHEN_IDLE.
Python module attribute vllm.envs.VLLM_TARGET_DEVICE.
Python module attribute vllm.envs.VLLM_TOOL_JSON_ERROR_AUTOMATIC_RETRY.
Python module attribute vllm.envs.VLLM_TOOL_PARSE_REGEX_TIMEOUT_SECONDS.
Python binding for vllm.envs.VLLM_TORCH_CUDA_PROFILE.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_DIR.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_DISABLE_ASYNC_LLM.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_DUMP_CUDA_TIME_TOTAL.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_RECORD_SHAPES.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_USE_GZIP.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_WITH_FLOPS.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_WITH_PROFILE_MEMORY.
Python binding for vllm.envs.VLLM_TORCH_PROFILER_WITH_STACK.
Python module attribute vllm.envs.VLLM_TPU_BUCKET_PADDING_GAP.
Python binding for vllm.envs.VLLM_TPU_MOST_MODEL_LEN.
Python module attribute vllm.envs.VLLM_TPU_USING_PATHWAYS.
Python module attribute vllm.envs.VLLM_TRACE_FUNCTION.
Python binding for vllm.envs.VLLM_TUNED_CONFIG_FOLDER.
Python module attribute vllm.envs.VLLM_USAGE_SOURCE.
Python module attribute vllm.envs.VLLM_USAGE_STATS_SERVER.
Python module attribute vllm.envs.VLLM_USE_AOT_COMPILE.
Python module attribute vllm.envs.VLLM_USE_BYTECODE_HOOK.
Python module attribute vllm.envs.VLLM_USE_DEEP_GEMM.
Python module attribute vllm.envs.VLLM_USE_DEEP_GEMM_E8M0.
Python binding for vllm.envs.VLLM_USE_DEEP_GEMM_TMA_ALIGNED_SCALES.
Python module attribute vllm.envs.VLLM_USE_EXPERIMENTAL_PARSER_CONTEXT.
Python module attribute vllm.envs.VLLM_USE_FBGEMM.
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_FP4.
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_FP8.
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_FP16.
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_MXFP4_BF16.
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8.
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8_CUTLASS.
Python binding for vllm.envs.VLLM_USE_FLASHINFER_SAMPLER.
Python module attribute vllm.envs.VLLM_USE_FUSED_MOE_GROUPED_TOPK.
Python binding for vllm.envs.VLLM_USE_MEGA_AOT_ARTIFACT.
Python module attribute vllm.envs.VLLM_USE_MODELSCOPE.
Python module attribute vllm.envs.VLLM_USE_NCCL_SYMM_MEM.
Python module attribute vllm.envs.VLLM_USE_NVFP4_CT_EMULATIONS.
Python module attribute vllm.envs.VLLM_USE_PRECOMPILED.
Python module attribute vllm.envs.VLLM_USE_RAY_COMPILED_DAG_CHANNEL_TYPE.
Python module attribute vllm.envs.VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM.
Python module attribute vllm.envs.VLLM_USE_RAY_WRAPPED_PP_COMM.
Python module attribute vllm.envs.VLLM_USE_STANDALONE_COMPILE.
Python module attribute vllm.envs.VLLM_USE_TRITON_AWQ.
Python module attribute vllm.envs.VLLM_USE_V2_MODEL_RUNNER.
Python module attribute vllm.envs.VLLM_V1_OUTPUT_PROC_CHUNK_SIZE.
Python module attribute vllm.envs.VLLM_V1_USE_OUTLINES_CACHE.
Python module attribute vllm.envs.VLLM_VIDEO_FETCH_TIMEOUT.
Python module attribute vllm.envs.VLLM_VIDEO_LOADER_BACKEND.
Python module attribute vllm.envs.VLLM_WORKER_MULTIPROC_METHOD.
Python module attribute vllm.envs.VLLM_XGRAMMAR_CACHE_MB.
Python module attribute vllm.envs.VLLM_XLA_CACHE_PATH.
Python module attribute vllm.envs.VLLM_XLA_CHECK_RECOMPILATION.
Python module attribute vllm.envs.VLLM_XLA_USE_SPMD.
Functions
@spec __dir__(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.__dir__.
Returns
term()
@spec __getattr__( String.t(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Gets environment variables lazily.
NOTE: After enable_envs_cache() invocation (which triggered after service initialization), all environment variables will be cached.
Parameters
name(String.t())
Returns
term()
@spec _is_envs_cache_enabled(keyword()) :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Checked if getattr is wrapped with functools.cache
Returns
boolean()
@spec cmake_build_type() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.CMAKE_BUILD_TYPE.
Returns
term()
@spec cmake_build_type(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cmake_build_type(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cmake_build_type( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cmake_build_type(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cmake_build_type(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cmake_build_type(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec compile_factors(keyword()) :: {:ok, %{optional(String.t()) => term()}} | {:error, Snakepit.Error.t()}
Return env vars used for torch.compile cache keys.
Start with every known vLLM env var; drop entries in ignored_factors;
hash everything else. This keeps the cache key aligned across workers.
Returns
%{optional(String.t()) => term()}
@spec cuda_visible_devices() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.CUDA_VISIBLE_DEVICES.
Returns
term()
@spec cuda_visible_devices(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cuda_visible_devices(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cuda_visible_devices( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cuda_visible_devices(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cuda_visible_devices(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec cuda_visible_devices(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec disable_compile_cache(keyword()) :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.disable_compile_cache.
Returns
boolean()
@spec disable_envs_cache(keyword()) :: {:ok, nil} | {:error, Snakepit.Error.t()}
Resets the environment variables cache. It could be used to isolate environments
between unit tests.
Returns
nil
@spec enable_envs_cache(keyword()) :: {:ok, nil} | {:error, Snakepit.Error.t()}
Enables caching of environment variables. This is useful for performance
reasons, as it avoids the need to re-evaluate environment variables on every call.
NOTE: Currently, it's invoked after service initialization to reduce runtime overhead. This also means that environment variables should NOT be updated after the service is initialized.
Returns
nil
@spec env_list_with_choices(String.t(), [String.t()], term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Create a lambda that validates environment variable
containing comma-separated values against allowed choices
Parameters
env_name- Name of the environment variabledefault- Default list of values if not setchoices- List of valid string options or callable that returns listcase_sensitive- Whether validation should be case sensitive
Returns
term()
@spec env_set_with_choices(String.t(), [String.t()], term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Creates a lambda which that validates environment variable
containing comma-separated values against allowed choices which returns choices as a set.
Parameters
env_name(String.t())default(list(String.t()))choices(term())case_sensitive(boolean() default: True)
Returns
term()
@spec env_with_choices(String.t(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Create a lambda that validates environment variable against allowed choices
Parameters
env_name- Name of the environment variabledefault- Default value if not set (can be None)choices- List of valid string options or callable that returns listcase_sensitive- Whether validation should be case sensitive
Returns
term()
@spec environment_variables() :: {:ok, %{optional(term()) => term()}} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.environment_variables.
Returns
%{optional(term()) => term()}
@spec get_default_cache_root(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.get_default_cache_root.
Returns
term()
@spec get_default_config_root(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.get_default_config_root.
Returns
term()
@spec get_env_or_set_default() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.get_env_or_set_default.
Returns
term()
@spec get_env_or_set_default(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec get_env_or_set_default(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec get_env_or_set_default( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec get_env_or_set_default(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec get_env_or_set_default(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec get_env_or_set_default(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec get_vllm_port(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Get the port from VLLM_PORT environment variable.
Raises
ArgumentError- If VLLM_PORT is a URI, suggest k8s service discovery issue.
Returns
term()
@spec is_set( String.t(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Check if an environment variable is explicitly set.
Parameters
name(String.t())
Returns
term()
@spec k_scale_constant() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.K_SCALE_CONSTANT.
Returns
integer()
@spec ld_library_path() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.LD_LIBRARY_PATH.
Returns
term()
@spec ld_library_path(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec ld_library_path(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec ld_library_path( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec ld_library_path(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec ld_library_path(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec ld_library_path(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec local_rank() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.LOCAL_RANK.
Returns
integer()
@spec logger() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.logger.
Returns
term()
@spec max_jobs() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.MAX_JOBS.
Returns
term()
@spec max_jobs(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec max_jobs(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec max_jobs( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec max_jobs(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec max_jobs(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec max_jobs(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec maybe_convert_bool( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.maybe_convert_bool.
Parameters
value(term())
Returns
term()
@spec maybe_convert_int( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.maybe_convert_int.
Parameters
value(term())
Returns
term()
@spec no_color() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.NO_COLOR.
Returns
boolean()
@spec nvcc_threads() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.NVCC_THREADS.
Returns
term()
@spec nvcc_threads(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec nvcc_threads(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec nvcc_threads( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec nvcc_threads(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec nvcc_threads(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec nvcc_threads(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec q_scale_constant() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.Q_SCALE_CONSTANT.
Returns
integer()
@spec s3_access_key_id() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.S3_ACCESS_KEY_ID.
Returns
term()
@spec s3_access_key_id(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_access_key_id(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_access_key_id( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_access_key_id(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_access_key_id(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_access_key_id(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_endpoint_url() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.S3_ENDPOINT_URL.
Returns
term()
@spec s3_endpoint_url(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_endpoint_url(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_endpoint_url( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_endpoint_url(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_endpoint_url(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_endpoint_url(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_secret_access_key() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.S3_SECRET_ACCESS_KEY.
Returns
term()
@spec s3_secret_access_key(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_secret_access_key(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_secret_access_key( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_secret_access_key(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_secret_access_key(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec s3_secret_access_key(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec use_aot_compile(keyword()) :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.use_aot_compile.
Returns
boolean()
@spec v_scale_constant() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.V_SCALE_CONSTANT.
Returns
integer()
@spec verbose() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VERBOSE.
Returns
boolean()
@spec vllm_all2_all_backend() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_ALL2ALL_BACKEND.
Returns
term()
@spec vllm_all2_all_backend(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_all2_all_backend(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_all2_all_backend( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_all2_all_backend(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_all2_all_backend(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_all2_all_backend(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_allow_chunked_local_attn_with_hybrid_kv_cache() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ALLOW_CHUNKED_LOCAL_ATTN_WITH_HYBRID_KV_CACHE.
Returns
boolean()
@spec vllm_allow_insecure_serialization() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ALLOW_INSECURE_SERIALIZATION.
Returns
boolean()
@spec vllm_allow_long_max_model_len() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ALLOW_LONG_MAX_MODEL_LEN.
Returns
boolean()
@spec vllm_allow_runtime_lora_updating() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ALLOW_RUNTIME_LORA_UPDATING.
Returns
boolean()
@spec vllm_allreduce_use_symm_mem() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ALLREDUCE_USE_SYMM_MEM.
Returns
boolean()
@spec vllm_api_key() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_API_KEY.
Returns
term()
@spec vllm_api_key(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_api_key(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_api_key( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_api_key(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_api_key(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_api_key(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_assets_cache() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ASSETS_CACHE.
Returns
String.t()
@spec vllm_assets_cache_model_clean() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ASSETS_CACHE_MODEL_CLEAN.
Returns
boolean()
@spec vllm_audio_fetch_timeout() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_AUDIO_FETCH_TIMEOUT.
Returns
integer()
@spec vllm_blockscale_fp8_gemm_flashinfer() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER.
Returns
boolean()
@spec vllm_cache_root() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_CACHE_ROOT.
Returns
String.t()
@spec vllm_compile_cache_save_format() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_COMPILE_CACHE_SAVE_FORMAT.
Returns
String.t()
@spec vllm_compute_nans_in_logits() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_COMPUTE_NANS_IN_LOGITS.
Returns
boolean()
@spec vllm_config_root() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_CONFIG_ROOT.
Returns
String.t()
@spec vllm_configure_logging() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_CONFIGURE_LOGGING.
Returns
boolean()
@spec vllm_cpu_kvcache_space() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_CPU_KVCACHE_SPACE.
Returns
term()
@spec vllm_cpu_kvcache_space(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_kvcache_space(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_kvcache_space( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_kvcache_space(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_kvcache_space(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_kvcache_space(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_CPU_NUM_OF_RESERVED_CPU.
Returns
term()
@spec vllm_cpu_num_of_reserved_cpu(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_num_of_reserved_cpu( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cpu_omp_threads_bind() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_CPU_OMP_THREADS_BIND.
Returns
String.t()
@spec vllm_cpu_sgl_kernel() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_CPU_SGL_KERNEL.
Returns
boolean()
@spec vllm_cudart_so_path() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_CUDART_SO_PATH.
Returns
term()
@spec vllm_cudart_so_path(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cudart_so_path(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cudart_so_path( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cudart_so_path(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cudart_so_path(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_cudart_so_path(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_custom_scopes_for_profiling() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_CUSTOM_SCOPES_FOR_PROFILING.
Returns
boolean()
@spec vllm_dbo_comm_sms() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DBO_COMM_SMS.
Returns
integer()
@spec vllm_debug_dump_path() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_DEBUG_DUMP_PATH.
Returns
term()
@spec vllm_debug_dump_path(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_debug_dump_path(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_debug_dump_path( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_debug_dump_path(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_debug_dump_path(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_debug_dump_path(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_debug_log_api_server_response() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEBUG_LOG_API_SERVER_RESPONSE.
Returns
boolean()
@spec vllm_debug_mfu_metrics() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEBUG_MFU_METRICS.
Returns
boolean()
@spec vllm_debug_workspace() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEBUG_WORKSPACE.
Returns
boolean()
@spec vllm_deep_gemm_warmup() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEEP_GEMM_WARMUP.
Returns
String.t()
@spec vllm_deepep_buffer_size_mb() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEEPEP_BUFFER_SIZE_MB.
Returns
integer()
@spec vllm_deepep_high_throughput_force_intra_node() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEEPEP_HIGH_THROUGHPUT_FORCE_INTRA_NODE.
Returns
boolean()
@spec vllm_deepep_low_latency_use_mnnvl() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEEPEP_LOW_LATENCY_USE_MNNVL.
Returns
boolean()
@spec vllm_deepepll_nvfp4_dispatch() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DEEPEPLL_NVFP4_DISPATCH.
Returns
boolean()
@spec vllm_disable_compile_cache() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DISABLE_COMPILE_CACHE.
Returns
boolean()
@spec vllm_disable_log_logo() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_DISABLE_LOG_LOGO.
Returns
term()
@spec vllm_disable_log_logo(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_disable_log_logo(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_disable_log_logo( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_disable_log_logo(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_disable_log_logo(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_disable_log_logo(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_disable_pynccl() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DISABLE_PYNCCL.
Returns
boolean()
@spec vllm_disabled_kernels() :: {:ok, [term()]} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DISABLED_KERNELS.
Returns
list(term())
@spec vllm_do_not_track() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DO_NOT_TRACK.
Returns
boolean()
@spec vllm_docker_build_context() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DOCKER_BUILD_CONTEXT.
Returns
boolean()
@spec vllm_dp_master_ip() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DP_MASTER_IP.
Returns
String.t()
@spec vllm_dp_master_port() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DP_MASTER_PORT.
Returns
integer()
@spec vllm_dp_rank() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DP_RANK.
Returns
integer()
@spec vllm_dp_rank_local() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DP_RANK_LOCAL.
Returns
integer()
@spec vllm_dp_size() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_DP_SIZE.
Returns
integer()
@spec vllm_enable_cudagraph_gc() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENABLE_CUDAGRAPH_GC.
Returns
boolean()
@spec vllm_enable_fused_moe_activation_chunking() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENABLE_FUSED_MOE_ACTIVATION_CHUNKING.
Returns
boolean()
@spec vllm_enable_inductor_coordinate_descent_tuning() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENABLE_INDUCTOR_COORDINATE_DESCENT_TUNING.
Returns
boolean()
@spec vllm_enable_inductor_max_autotune() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE.
Returns
boolean()
@spec vllm_enable_moe_dp_chunk() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENABLE_MOE_DP_CHUNK.
Returns
boolean()
@spec vllm_enable_responses_api_store() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENABLE_RESPONSES_API_STORE.
Returns
boolean()
@spec vllm_enable_v1_multiprocessing() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENABLE_V1_MULTIPROCESSING.
Returns
boolean()
@spec vllm_engine_iteration_timeout_s() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENGINE_ITERATION_TIMEOUT_S.
Returns
integer()
@spec vllm_engine_ready_timeout_s() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ENGINE_READY_TIMEOUT_S.
Returns
integer()
@spec vllm_execute_model_timeout_seconds() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS.
Returns
integer()
@spec vllm_flashinfer_moe_backend() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_FLASHINFER_MOE_BACKEND.
Returns
String.t()
@spec vllm_flashinfer_workspace_buffer_size() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE.
Returns
integer()
@spec vllm_float32_matmul_precision() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_FLOAT32_MATMUL_PRECISION.
Returns
String.t()
@spec vllm_force_aot_load() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_FORCE_AOT_LOAD.
Returns
boolean()
@spec vllm_fused_moe_chunk_size() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_FUSED_MOE_CHUNK_SIZE.
Returns
integer()
@spec vllm_gc_debug() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_GC_DEBUG.
Returns
String.t()
@spec vllm_gpt_oss_harmony_system_instructions() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_GPT_OSS_HARMONY_SYSTEM_INSTRUCTIONS.
Returns
boolean()
@spec vllm_gpt_oss_system_tool_mcp_labels() :: {:ok, MapSet.t(term())} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_GPT_OSS_SYSTEM_TOOL_MCP_LABELS.
Returns
MapSet.t(term())
@spec vllm_has_flashinfer_cubin() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_HAS_FLASHINFER_CUBIN.
Returns
boolean()
@spec vllm_host_ip() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_HOST_IP.
Returns
String.t()
@spec vllm_http_timeout_keep_alive() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_HTTP_TIMEOUT_KEEP_ALIVE.
Returns
integer()
@spec vllm_image_fetch_timeout() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_IMAGE_FETCH_TIMEOUT.
Returns
integer()
@spec vllm_keep_alive_on_engine_death() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_KEEP_ALIVE_ON_ENGINE_DEATH.
Returns
boolean()
@spec vllm_kv_cache_layout() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_KV_CACHE_LAYOUT.
Returns
term()
@spec vllm_kv_cache_layout(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_kv_cache_layout(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_kv_cache_layout( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_kv_cache_layout(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_kv_cache_layout(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_kv_cache_layout(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_kv_events_use_int_block_hashes() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_KV_EVENTS_USE_INT_BLOCK_HASHES.
Returns
boolean()
@spec vllm_log_batchsize_interval() :: {:ok, float()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOG_BATCHSIZE_INTERVAL.
Returns
float()
@spec vllm_log_model_inspection() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOG_MODEL_INSPECTION.
Returns
boolean()
@spec vllm_log_stats_interval() :: {:ok, float()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOG_STATS_INTERVAL.
Returns
float()
@spec vllm_logging_color() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOGGING_COLOR.
Returns
String.t()
@spec vllm_logging_config_path() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_LOGGING_CONFIG_PATH.
Returns
term()
@spec vllm_logging_config_path(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_logging_config_path(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_logging_config_path( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_logging_config_path(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_logging_config_path(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_logging_config_path(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_logging_level() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOGGING_LEVEL.
Returns
String.t()
@spec vllm_logging_prefix() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOGGING_PREFIX.
Returns
String.t()
@spec vllm_logging_stream() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOGGING_STREAM.
Returns
String.t()
@spec vllm_loopback_ip() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_LOOPBACK_IP.
Returns
String.t()
@spec vllm_lora_resolver_cache_dir() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_LORA_RESOLVER_CACHE_DIR.
Returns
term()
@spec vllm_lora_resolver_cache_dir(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_lora_resolver_cache_dir(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_lora_resolver_cache_dir( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_lora_resolver_cache_dir(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_lora_resolver_cache_dir(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_lora_resolver_cache_dir(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_lora_resolver_cache_dir( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_lora_resolver_cache_dir( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_main_cuda_version() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MAIN_CUDA_VERSION.
Returns
String.t()
@spec vllm_marlin_input_dtype() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_MARLIN_INPUT_DTYPE.
Returns
term()
@spec vllm_marlin_input_dtype(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_marlin_input_dtype(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_marlin_input_dtype( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_marlin_input_dtype(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_marlin_input_dtype(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_marlin_input_dtype(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_marlin_use_atomic_add() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MARLIN_USE_ATOMIC_ADD.
Returns
boolean()
@spec vllm_max_audio_clip_filesize_mb() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MAX_AUDIO_CLIP_FILESIZE_MB.
Returns
integer()
@spec vllm_max_tokens_per_expert_fp4_moe() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE.
Returns
integer()
@spec vllm_media_connector() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MEDIA_CONNECTOR.
Returns
String.t()
@spec vllm_media_loading_thread_count() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MEDIA_LOADING_THREAD_COUNT.
Returns
integer()
@spec vllm_media_url_allow_redirects() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MEDIA_URL_ALLOW_REDIRECTS.
Returns
boolean()
@spec vllm_mla_disable() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MLA_DISABLE.
Returns
boolean()
@spec vllm_mm_hasher_algorithm() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_MM_HASHER_ALGORITHM.
Returns
term()
@spec vllm_mm_hasher_algorithm(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mm_hasher_algorithm(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mm_hasher_algorithm( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mm_hasher_algorithm(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mm_hasher_algorithm(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mm_hasher_algorithm(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_model_redirect_path() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_MODEL_REDIRECT_PATH.
Returns
term()
@spec vllm_model_redirect_path(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_model_redirect_path(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_model_redirect_path( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_model_redirect_path(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_model_redirect_path(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_model_redirect_path(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_moe_dp_chunk_size() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MOE_DP_CHUNK_SIZE.
Returns
integer()
@spec vllm_moe_use_deep_gemm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MOE_USE_DEEP_GEMM.
Returns
boolean()
@spec vllm_mooncake_abort_request_timeout() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MOONCAKE_ABORT_REQUEST_TIMEOUT.
Returns
integer()
@spec vllm_mooncake_bootstrap_port() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MOONCAKE_BOOTSTRAP_PORT.
Returns
integer()
@spec vllm_moriio_connector_read_mode() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MORIIO_CONNECTOR_READ_MODE.
Returns
boolean()
@spec vllm_moriio_num_workers() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MORIIO_NUM_WORKERS.
Returns
integer()
@spec vllm_moriio_post_batch_size() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MORIIO_POST_BATCH_SIZE.
Returns
integer()
@spec vllm_moriio_qp_per_transfer() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MORIIO_QP_PER_TRANSFER.
Returns
integer()
@spec vllm_mq_max_chunk_bytes_mb() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MQ_MAX_CHUNK_BYTES_MB.
Returns
integer()
@spec vllm_msgpack_zero_copy_threshold() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_MSGPACK_ZERO_COPY_THRESHOLD.
Returns
integer()
@spec vllm_mxfp4_use_marlin() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_MXFP4_USE_MARLIN.
Returns
term()
@spec vllm_mxfp4_use_marlin(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mxfp4_use_marlin(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mxfp4_use_marlin( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mxfp4_use_marlin(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mxfp4_use_marlin(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_mxfp4_use_marlin(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_include_path() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_NCCL_INCLUDE_PATH.
Returns
term()
@spec vllm_nccl_include_path(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_include_path(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_include_path( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_include_path(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_include_path(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_include_path(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_so_path() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_NCCL_SO_PATH.
Returns
term()
@spec vllm_nccl_so_path(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_so_path(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_so_path( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_so_path(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_so_path(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nccl_so_path(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nixl_abort_request_timeout() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_NIXL_ABORT_REQUEST_TIMEOUT.
Returns
integer()
@spec vllm_nixl_side_channel_host() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_NIXL_SIDE_CHANNEL_HOST.
Returns
String.t()
@spec vllm_nixl_side_channel_port() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_NIXL_SIDE_CHANNEL_PORT.
Returns
integer()
@spec vllm_no_usage_stats() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_NO_USAGE_STATS.
Returns
boolean()
@spec vllm_nvfp4_gemm_backend() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_NVFP4_GEMM_BACKEND.
Returns
term()
@spec vllm_nvfp4_gemm_backend(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nvfp4_gemm_backend(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nvfp4_gemm_backend( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nvfp4_gemm_backend(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nvfp4_gemm_backend(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nvfp4_gemm_backend(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_nvtx_scopes_for_profiling() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_NVTX_SCOPES_FOR_PROFILING.
Returns
boolean()
@spec vllm_object_storage_shm_buffer_name() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_OBJECT_STORAGE_SHM_BUFFER_NAME.
Returns
String.t()
@spec vllm_pattern_match_debug() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_PATTERN_MATCH_DEBUG.
Returns
term()
@spec vllm_pattern_match_debug(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pattern_match_debug(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pattern_match_debug( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pattern_match_debug(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pattern_match_debug(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pattern_match_debug(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_plugins() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_PLUGINS.
Returns
term()
@spec vllm_plugins(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_plugins(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_plugins( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_plugins(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_plugins(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_plugins(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_port() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_PORT.
Returns
term()
@spec vllm_port(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_port(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_port( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_port(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_port(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_port(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pp_layer_partition() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_PP_LAYER_PARTITION.
Returns
term()
@spec vllm_pp_layer_partition(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pp_layer_partition(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pp_layer_partition( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pp_layer_partition(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pp_layer_partition(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_pp_layer_partition(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_delay_iters() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_PROFILER_DELAY_ITERS.
Returns
term()
@spec vllm_profiler_delay_iters(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_delay_iters(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_delay_iters( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_delay_iters(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_delay_iters(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_delay_iters(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_max_iters() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_PROFILER_MAX_ITERS.
Returns
term()
@spec vllm_profiler_max_iters(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_max_iters(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_max_iters( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_max_iters(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_max_iters(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_profiler_max_iters(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_randomize_dp_dummy_inputs() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_RANDOMIZE_DP_DUMMY_INPUTS.
Returns
boolean()
@spec vllm_ray_bundle_indices() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_RAY_BUNDLE_INDICES.
Returns
String.t()
@spec vllm_ray_dp_pack_strategy() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_RAY_DP_PACK_STRATEGY.
Returns
String.t()
@spec vllm_ray_per_worker_gpus() :: {:ok, float()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_RAY_PER_WORKER_GPUS.
Returns
float()
@spec vllm_ringbuffer_warning_interval() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_RINGBUFFER_WARNING_INTERVAL.
Returns
integer()
@spec vllm_rocm_custom_paged_attn() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_CUSTOM_PAGED_ATTN.
Returns
boolean()
@spec vllm_rocm_fp8_mfma_page_attn() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_FP8_MFMA_PAGE_ATTN.
Returns
boolean()
@spec vllm_rocm_fp8_padding() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_FP8_PADDING.
Returns
boolean()
@spec vllm_rocm_moe_padding() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_MOE_PADDING.
Returns
boolean()
@spec vllm_rocm_quick_reduce_cast_bf16_to_fp16() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_QUICK_REDUCE_CAST_BF16_TO_FP16.
Returns
boolean()
@spec vllm_rocm_quick_reduce_max_size_bytes_mb() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_ROCM_QUICK_REDUCE_MAX_SIZE_BYTES_MB.
Returns
term()
@spec vllm_rocm_quick_reduce_max_size_bytes_mb(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb( term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb( term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_max_size_bytes_mb( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_quick_reduce_quantization() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_QUICK_REDUCE_QUANTIZATION.
Returns
String.t()
@spec vllm_rocm_shuffle_kv_cache_layout() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT.
Returns
term()
@spec vllm_rocm_shuffle_kv_cache_layout(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_shuffle_kv_cache_layout(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_shuffle_kv_cache_layout( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_shuffle_kv_cache_layout(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_shuffle_kv_cache_layout(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_shuffle_kv_cache_layout(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_shuffle_kv_cache_layout( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_shuffle_kv_cache_layout( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_sleep_mem_chunk_size() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_SLEEP_MEM_CHUNK_SIZE.
Returns
integer()
@spec vllm_rocm_use_aiter() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER.
Returns
boolean()
@spec vllm_rocm_use_aiter_fp4_asm_gemm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_FP4_ASM_GEMM.
Returns
boolean()
@spec vllm_rocm_use_aiter_fp4_bmm() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_ROCM_USE_AITER_FP4BMM.
Returns
term()
@spec vllm_rocm_use_aiter_fp4_bmm(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_use_aiter_fp4_bmm(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_use_aiter_fp4_bmm( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_use_aiter_fp4_bmm(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_use_aiter_fp4_bmm(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_use_aiter_fp4_bmm(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_rocm_use_aiter_fp8_bmm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_FP8BMM.
Returns
boolean()
@spec vllm_rocm_use_aiter_linear() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_LINEAR.
Returns
boolean()
@spec vllm_rocm_use_aiter_mha() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_MHA.
Returns
boolean()
@spec vllm_rocm_use_aiter_mla() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_MLA.
Returns
boolean()
@spec vllm_rocm_use_aiter_moe() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_MOE.
Returns
boolean()
@spec vllm_rocm_use_aiter_paged_attn() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_PAGED_ATTN.
Returns
boolean()
@spec vllm_rocm_use_aiter_rmsnorm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_RMSNORM.
Returns
boolean()
@spec vllm_rocm_use_aiter_triton_gemm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_TRITON_GEMM.
Returns
boolean()
@spec vllm_rocm_use_aiter_triton_rope() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_TRITON_ROPE.
Returns
boolean()
@spec vllm_rocm_use_aiter_unified_attention() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION.
Returns
boolean()
@spec vllm_rocm_use_skinny_gemm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_ROCM_USE_SKINNY_GEMM.
Returns
boolean()
@spec vllm_rpc_base_path() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_RPC_BASE_PATH.
Returns
String.t()
@spec vllm_rpc_timeout() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_RPC_TIMEOUT.
Returns
integer()
@spec vllm_server_dev_mode() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_SERVER_DEV_MODE.
Returns
boolean()
@spec vllm_skip_p2_p_check() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_SKIP_P2P_CHECK.
Returns
boolean()
@spec vllm_skip_precompiled_version_suffix() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_SKIP_PRECOMPILED_VERSION_SUFFIX.
Returns
boolean()
@spec vllm_sleep_when_idle() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_SLEEP_WHEN_IDLE.
Returns
boolean()
@spec vllm_target_device() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_TARGET_DEVICE.
Returns
String.t()
@spec vllm_tool_json_error_automatic_retry() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_TOOL_JSON_ERROR_AUTOMATIC_RETRY.
Returns
boolean()
@spec vllm_tool_parse_regex_timeout_seconds() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_TOOL_PARSE_REGEX_TIMEOUT_SECONDS.
Returns
integer()
@spec vllm_torch_cuda_profile() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_CUDA_PROFILE.
Returns
term()
@spec vllm_torch_cuda_profile(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_cuda_profile(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_cuda_profile( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_cuda_profile(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_cuda_profile(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_cuda_profile(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dir() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_DIR.
Returns
term()
@spec vllm_torch_profiler_dir(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dir(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dir( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dir(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dir(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dir(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_DISABLE_ASYNC_LLM.
Returns
term()
@spec vllm_torch_profiler_disable_async_llm(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm( term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm( term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_disable_async_llm( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_DUMP_CUDA_TIME_TOTAL.
Returns
term()
@spec vllm_torch_profiler_dump_cuda_time_total(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total( term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total( term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_dump_cuda_time_total( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_RECORD_SHAPES.
Returns
term()
@spec vllm_torch_profiler_record_shapes(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_record_shapes( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_USE_GZIP.
Returns
term()
@spec vllm_torch_profiler_use_gzip(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_use_gzip( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_WITH_FLOPS.
Returns
term()
@spec vllm_torch_profiler_with_flops(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_flops( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_WITH_PROFILE_MEMORY.
Returns
term()
@spec vllm_torch_profiler_with_profile_memory(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory( term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory( term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_profile_memory( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TORCH_PROFILER_WITH_STACK.
Returns
term()
@spec vllm_torch_profiler_with_stack(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_torch_profiler_with_stack( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tpu_bucket_padding_gap() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_TPU_BUCKET_PADDING_GAP.
Returns
integer()
@spec vllm_tpu_most_model_len() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TPU_MOST_MODEL_LEN.
Returns
term()
@spec vllm_tpu_most_model_len(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tpu_most_model_len(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tpu_most_model_len( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tpu_most_model_len(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tpu_most_model_len(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tpu_most_model_len(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tpu_using_pathways() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_TPU_USING_PATHWAYS.
Returns
boolean()
@spec vllm_trace_function() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_TRACE_FUNCTION.
Returns
integer()
@spec vllm_tuned_config_folder() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_TUNED_CONFIG_FOLDER.
Returns
term()
@spec vllm_tuned_config_folder(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tuned_config_folder(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tuned_config_folder( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tuned_config_folder(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tuned_config_folder(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_tuned_config_folder(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_usage_source() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USAGE_SOURCE.
Returns
String.t()
@spec vllm_usage_stats_server() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USAGE_STATS_SERVER.
Returns
String.t()
@spec vllm_use_aot_compile() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_AOT_COMPILE.
Returns
boolean()
@spec vllm_use_bytecode_hook() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_BYTECODE_HOOK.
Returns
boolean()
@spec vllm_use_deep_gemm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_DEEP_GEMM.
Returns
boolean()
@spec vllm_use_deep_gemm_e8_m0() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_DEEP_GEMM_E8M0.
Returns
boolean()
@spec vllm_use_deep_gemm_tma_aligned_scales() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_USE_DEEP_GEMM_TMA_ALIGNED_SCALES.
Returns
term()
@spec vllm_use_deep_gemm_tma_aligned_scales(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales( term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales( term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales( term(), term(), term(), term(), term(), term(), term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_deep_gemm_tma_aligned_scales( term(), term(), term(), term(), term(), term(), term(), term() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_experimental_parser_context() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_EXPERIMENTAL_PARSER_CONTEXT.
Returns
boolean()
@spec vllm_use_fbgemm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FBGEMM.
Returns
boolean()
@spec vllm_use_flashinfer_moe_fp4() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_FP4.
Returns
boolean()
@spec vllm_use_flashinfer_moe_fp8() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_FP8.
Returns
boolean()
@spec vllm_use_flashinfer_moe_fp16() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_FP16.
Returns
boolean()
@spec vllm_use_flashinfer_moe_mxfp4_bf16() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_MXFP4_BF16.
Returns
boolean()
@spec vllm_use_flashinfer_moe_mxfp4_mxfp8() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8.
Returns
boolean()
@spec vllm_use_flashinfer_moe_mxfp4_mxfp8_cutlass() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8_CUTLASS.
Returns
boolean()
@spec vllm_use_flashinfer_sampler() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_USE_FLASHINFER_SAMPLER.
Returns
term()
@spec vllm_use_flashinfer_sampler(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_flashinfer_sampler(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_flashinfer_sampler( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_flashinfer_sampler(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_flashinfer_sampler(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_flashinfer_sampler(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_fused_moe_grouped_topk() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_FUSED_MOE_GROUPED_TOPK.
Returns
boolean()
@spec vllm_use_mega_aot_artifact() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.envs.VLLM_USE_MEGA_AOT_ARTIFACT.
Returns
term()
@spec vllm_use_mega_aot_artifact(keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_mega_aot_artifact(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_mega_aot_artifact( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_mega_aot_artifact(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_mega_aot_artifact(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_mega_aot_artifact(term(), term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec vllm_use_modelscope() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_MODELSCOPE.
Returns
boolean()
@spec vllm_use_nccl_symm_mem() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_NCCL_SYMM_MEM.
Returns
boolean()
@spec vllm_use_nvfp4_ct_emulations() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_NVFP4_CT_EMULATIONS.
Returns
boolean()
@spec vllm_use_precompiled() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_PRECOMPILED.
Returns
boolean()
@spec vllm_use_ray_compiled_dag_channel_type() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_RAY_COMPILED_DAG_CHANNEL_TYPE.
Returns
String.t()
@spec vllm_use_ray_compiled_dag_overlap_comm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM.
Returns
boolean()
@spec vllm_use_ray_wrapped_pp_comm() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_RAY_WRAPPED_PP_COMM.
Returns
boolean()
@spec vllm_use_standalone_compile() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_STANDALONE_COMPILE.
Returns
boolean()
@spec vllm_use_triton_awq() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_TRITON_AWQ.
Returns
boolean()
@spec vllm_use_v2_model_runner() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_USE_V2_MODEL_RUNNER.
Returns
boolean()
@spec vllm_v1_output_proc_chunk_size() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_V1_OUTPUT_PROC_CHUNK_SIZE.
Returns
integer()
@spec vllm_v1_use_outlines_cache() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_V1_USE_OUTLINES_CACHE.
Returns
boolean()
@spec vllm_video_fetch_timeout() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_VIDEO_FETCH_TIMEOUT.
Returns
integer()
@spec vllm_video_loader_backend() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_VIDEO_LOADER_BACKEND.
Returns
String.t()
@spec vllm_worker_multiproc_method() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_WORKER_MULTIPROC_METHOD.
Returns
String.t()
@spec vllm_xgrammar_cache_mb() :: {:ok, integer()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_XGRAMMAR_CACHE_MB.
Returns
integer()
@spec vllm_xla_cache_path() :: {:ok, String.t()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_XLA_CACHE_PATH.
Returns
String.t()
@spec vllm_xla_check_recompilation() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_XLA_CHECK_RECOMPILATION.
Returns
boolean()
@spec vllm_xla_use_spmd() :: {:ok, boolean()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.envs.VLLM_XLA_USE_SPMD.
Returns
boolean()