Vllm.Logprobs.Logprob (VLLM v0.3.0)

Copy Markdown View Source

Infos for supporting OpenAI compatible logprobs and token ranks.

Attributes:

logprob: The logprob of chosen token
rank: The vocab rank of chosen token (>=1)
decoded_token: The decoded chosen token index

Summary

Types

t()

Functions

decoded_token(ref)
new(logprob, args, opts \\ [])

Initialize self. See help(type(self)) for accurate signature.

rank(ref)

Types

t()

@opaque t()

Functions

decoded_token(ref)

@spec decoded_token(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

new(logprob, args, opts \\ [])

@spec new(float(), [term()], keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Initialize self. See help(type(self)) for accurate signature.

Parameters

  • logprob (float())
  • rank (term() default: None)
  • decoded_token (term() default: None)

rank(ref)

@spec rank(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}