Vllm.Outputs.RequestOutput (VLLM v0.3.0)

Copy Markdown View Source

The output data of a completion request to the LLM.

Parameters

  • request_id - The unique ID of the request.
  • prompt - The prompt string of the request. For encoder/decoder models, this is the decoder input prompt.
  • prompt_token_ids - The token IDs of the prompt. For encoder/decoder models, this is the decoder input prompt token ids.
  • prompt_logprobs - The log probabilities to return per prompt token.
  • outputs - The output sequences of the request.
  • finished - Whether the whole request is finished.
  • metrics - Metrics associated with the request.
  • lora_request - The LoRA request that was used to generate the output.
  • encoder_prompt - The encoder prompt string of the request. None if decoder-only.
  • encoder_prompt_token_ids - The token IDs of the encoder prompt. None if decoder-only.
  • num_cached_tokens - The number of tokens with prefix cache hit.
  • kv_transfer_params - The params for remote K/V transfer.

Summary

Types

t()

Functions

add(ref, next_output, aggregate, opts \\ [])

Merge subsequent RequestOutput into this one

new(request_id, prompt, prompt_token_ids, prompt_logprobs, outputs, finished, args, opts \\ [])

Initialize self. See help(type(self)) for accurate signature.

Types

t()

@opaque t()

Functions

add(ref, next_output, aggregate, opts \\ [])

@spec add(SnakeBridge.Ref.t(), t(), boolean(), keyword()) ::
  {:ok, nil} | {:error, Snakepit.Error.t()}

Merge subsequent RequestOutput into this one

Parameters

  • next_output (Vllm.Outputs.RequestOutput.t())
  • aggregate (boolean())

Returns

  • nil

new(request_id, prompt, prompt_token_ids, prompt_logprobs, outputs, finished, args, opts \\ [])

@spec new(
  String.t(),
  term(),
  term(),
  term(),
  [Vllm.Outputs.CompletionOutput.t()],
  boolean(),
  [term()],
  keyword()
) :: {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Initialize self. See help(type(self)) for accurate signature.

Parameters

  • request_id (String.t())
  • prompt (term())
  • prompt_token_ids (term())
  • prompt_logprobs (term())
  • outputs (list(Vllm.Outputs.CompletionOutput.t()))
  • finished (boolean())
  • metrics (term() default: None)
  • lora_request (term() default: None)
  • encoder_prompt (term() default: None)
  • encoder_prompt_token_ids (term() default: None)
  • num_cached_tokens (term() default: None)
  • multi_modal_placeholders (term() keyword-only default: None)
  • kv_transfer_params (term() keyword-only default: None)
  • kwargs (term())