Vllm.BeamSearch.BeamSearchSequence (VLLM v0.3.0)

Copy Markdown View Source

A sequence for beam search.

It keeps track of the tokens and the log probability of the sequence. The text field is optional and will only be filled when the sequence is about to be returned to the user.

Summary

Types

t()

Functions

cum_logprob(ref)
finish_reason(ref)
lora_request(ref)
mm_processor_kwargs(ref)
multi_modal_data(ref)
new(tokens, logprobs, args, opts \\ [])

Initialize self. See help(type(self)) for accurate signature.

stop_reason(ref)
text(ref)

Types

t()

@opaque t()

Functions

cum_logprob(ref)

@spec cum_logprob(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

finish_reason(ref)

@spec finish_reason(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

lora_request(ref)

@spec lora_request(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

mm_processor_kwargs(ref)

@spec mm_processor_kwargs(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

multi_modal_data(ref)

@spec multi_modal_data(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

new(tokens, logprobs, args, opts \\ [])

@spec new(
  [integer()],
  [%{optional(integer()) => Vllm.Logprobs.Logprob.t()}],
  [term()],
  keyword()
) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Initialize self. See help(type(self)) for accurate signature.

Parameters

  • tokens (list(integer()))
  • logprobs (list(%{optional(integer()) => Vllm.Logprobs.Logprob.t()}))
  • lora_request (term() default: None)
  • cum_logprob (float() default: 0.0)
  • text (term() default: None)
  • finish_reason (term() default: None)
  • stop_reason (term() default: None)

  • multi_modal_data (term() | nil default: None)

  • mm_processor_kwargs (term() default: None)

stop_reason(ref)

@spec stop_reason(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

text(ref)

@spec text(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}