Submodule bindings for vllm.inputs.
Version
- Requested: 0.14.0
- Observed at generation: 0.14.0
Runtime Options
All functions accept a __runtime__ option for controlling execution behavior:
Vllm.Inputs.some_function(args, __runtime__: [timeout: 120_000])Supported runtime options
:timeout- Call timeout in milliseconds (default: 120,000ms / 2 minutes):timeout_profile- Use a named profile (:default,:ml_inference,:batch_job,:streaming):stream_timeout- Timeout for streaming operations (default: 1,800,000ms / 30 minutes):session_id- Override the session ID for this call:pool_name- Target a specific Snakepit pool (multi-pool setups):affinity- Override session affinity (:hint,:strict_queue,:strict_fail_fast)
Timeout Profiles
:default- 2 minute timeout for regular calls:ml_inference- 10 minute timeout for ML/LLM workloads:batch_job- Unlimited timeout for long-running jobs:streaming- 2 minute timeout, 30 minute stream_timeout
Example with timeout override
# For a long-running ML inference call
Vllm.Inputs.predict(data, __runtime__: [timeout_profile: :ml_inference])
# Or explicit timeout
Vllm.Inputs.predict(data, __runtime__: [timeout: 600_000])
# Route to a pool and enforce strict affinity
Vllm.Inputs.predict(data, __runtime__: [pool_name: :strict_pool, affinity: :strict_queue])See SnakeBridge.Defaults for global timeout configuration.
Summary
Functions
Python module attribute vllm.inputs.__all__.
Python binding for vllm.inputs.build_explicit_enc_dec_prompt.
Python module attribute vllm.inputs.DecoderOnlyInputs.
Construct [EmbedsInputs][vllm.inputs.data.EmbedsInputs] from optional
Python module attribute vllm.inputs.ProcessorInputs.
Python module attribute vllm.inputs.PromptType.
Python module attribute vllm.inputs.SingletonInputs.
Python module attribute vllm.inputs.SingletonPrompt.
Python binding for vllm.inputs.to_enc_dec_tuple_list.
Construct [TokenInputs][vllm.inputs.data.TokenInputs] from optional
Zip encoder and decoder prompts together into a list of
Functions
@spec __all__() :: {:ok, [term()]} | {:error, Snakepit.Error.t()}
Python module attribute vllm.inputs.__all__.
Returns
list(term())
@spec build_explicit_enc_dec_prompt(term(), term() | nil) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python binding for vllm.inputs.build_explicit_enc_dec_prompt.
Parameters
encoder_prompt(term())decoder_prompt(term() | nil)mm_processor_kwargs(term() default: None)
Returns
term()
@spec build_explicit_enc_dec_prompt(term(), term() | nil, keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec build_explicit_enc_dec_prompt(term(), term() | nil, term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec decoder_only_inputs() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.inputs.DecoderOnlyInputs.
Returns
term()
@spec embeds_inputs(term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Construct [EmbedsInputs][vllm.inputs.data.EmbedsInputs] from optional
values.
Parameters
prompt_embeds(term())cache_salt(term() default: None)
Returns
term()
@spec embeds_inputs( term(), keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec embeds_inputs(term(), term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec embeds_inputs(term(), term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec processor_inputs() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.inputs.ProcessorInputs.
Returns
term()
@spec prompt_type() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.inputs.PromptType.
Returns
term()
@spec singleton_inputs() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.inputs.SingletonInputs.
Returns
term()
@spec singleton_prompt() :: {:ok, term()} | {:error, Snakepit.Error.t()}
Python module attribute vllm.inputs.SingletonPrompt.
Returns
term()
@spec to_enc_dec_tuple_list( term(), keyword() ) :: {:ok, [{term(), term() | nil}]} | {:error, Snakepit.Error.t()}
Python binding for vllm.inputs.to_enc_dec_tuple_list.
Parameters
enc_dec_prompts(term())
Returns
list({term(), term() | nil})
@spec token_inputs([integer()]) :: {:ok, term()} | {:error, Snakepit.Error.t()}
Construct [TokenInputs][vllm.inputs.data.TokenInputs] from optional
values.
Parameters
prompt_token_ids(list(integer()))cache_salt(term() default: None)
Returns
term()
@spec token_inputs( [integer()], keyword() ) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec token_inputs([integer()], term()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec token_inputs([integer()], term(), keyword()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
@spec zip_enc_dec_prompts(term(), term()) :: {:ok, [term()]} | {:error, Snakepit.Error.t()}
Zip encoder and decoder prompts together into a list of
[ExplicitEncoderDecoderPrompt][vllm.inputs.data.ExplicitEncoderDecoderPrompt]
instances.
mm_processor_kwargs may also be provided; if a dict is passed, the same
dictionary will be used for every encoder/decoder prompt. If an iterable is
provided, it will be zipped with the encoder/decoder prompts.
Parameters
enc_prompts(term())dec_prompts(term())mm_processor_kwargs(term() default: None)
Returns
list(term())
@spec zip_enc_dec_prompts(term(), term(), keyword()) :: {:ok, [term()]} | {:error, Snakepit.Error.t()}
@spec zip_enc_dec_prompts(term(), term(), term()) :: {:ok, [term()]} | {:error, Snakepit.Error.t()}