Vllm.Multimodal.Inputs.PlaceholderRange (VLLM v0.3.0)

Copy Markdown View Source

Placeholder location information for multi-modal data.

Summary

Functions

Extract the start and end indices of the embedded region in prompt.

Returns the starting and ending indices of the embeddings of encoder outputs

Initialize self. See help(type(self)) for accurate signature.

Types

t()

@opaque t()

Functions

embeds_cumsum(ref)

@spec embeds_cumsum(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

extract_embeds_range(ref, opts \\ [])

@spec extract_embeds_range(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, [{integer(), integer()}]} | {:error, Snakepit.Error.t()}

Extract the start and end indices of the embedded region in prompt.

For example, given PlaceholderRange(offset=2, length=5) and is_embed = [False, True, False, True, True], the output is [(1 + offset, 1 + offset), (3 + offset, 4 + offset)].

Returns

  • list({integer(), integer()})

get_embeds_indices_in_range(ref, start_idx, end_idx, opts \\ [])

@spec get_embeds_indices_in_range(
  SnakeBridge.Ref.t(),
  integer(),
  integer(),
  keyword()
) ::
  {:ok, {integer(), integer()}} | {:error, Snakepit.Error.t()}

Returns the starting and ending indices of the embeddings of encoder outputs

in the range of [start_idx, end_idx) in the placeholders.

For example, given: PlaceholderRange(offset=2, length=5, is_embed=[False, True, False, True, True])

If start_idx=3 and end_idx=5, the output is (1, 3) because we want to get the second and the third embeddings from the encoder output.

Parameters

  • start_idx (integer())
  • end_idx (integer())

Returns

  • {integer(), integer()}

get_num_embeds(ref)

@spec get_num_embeds(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

is_embed(ref)

@spec is_embed(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}

new(offset, length, args, opts \\ [])

@spec new(integer(), integer(), [term()], keyword()) ::
  {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Initialize self. See help(type(self)) for accurate signature.

Parameters

  • offset (integer())
  • length (integer())
  • is_embed (term() | nil default: None)