Vllm.ToolParsers.ToolParser (VLLM v0.3.0)

Copy Markdown View Source

Abstract ToolParser class that should not be used directly. Provided

properties and methods should be used in derived classes.

Summary

Functions

Static method that used to adjust the request parameters.

Static method that should be implemented for extracting tool calls from

Initialize self. See help(type(self)) for accurate signature.

Types

t()

@opaque t()

Functions

adjust_request(ref, request, opts \\ [])

@spec adjust_request(SnakeBridge.Ref.t(), term(), keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

Static method that used to adjust the request parameters.

Parameters

  • request (term())

Returns

  • term()

extract_tool_calls(ref, model_output, request, opts \\ [])

@spec extract_tool_calls(SnakeBridge.Ref.t(), String.t(), term(), keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}

Static method that should be implemented for extracting tool calls from

a complete model-generated string. Used for non-streaming responses where we have the entire model response available before sending to the client. Static because it's stateless.

Parameters

  • model_output (String.t())
  • request (term())

Returns

  • term()

extract_tool_calls_streaming(ref, previous_text, current_text, delta_text, previous_token_ids, current_token_ids, delta_token_ids, request, opts \\ [])

@spec extract_tool_calls_streaming(
  SnakeBridge.Ref.t(),
  String.t(),
  String.t(),
  String.t(),
  term(),
  term(),
  term(),
  term(),
  keyword()
) :: {:ok, term()} | {:error, Snakepit.Error.t()}

Instance method that should be implemented for extracting tool calls

from an incomplete response; for use when handling tool calls and streaming. Has to be an instance method because it requires state - the current tokens/diffs, but also the information about what has previously been parsed and extracted (see constructor)

Parameters

  • previous_text (String.t())
  • current_text (String.t())
  • delta_text (String.t())
  • previous_token_ids (term())
  • current_token_ids (term())
  • delta_token_ids (term())
  • request (term())

Returns

  • term()

new(tokenizer, opts \\ [])

@spec new(
  term(),
  keyword()
) :: {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}

Initialize self. See help(type(self)) for accurate signature.

Parameters

  • tokenizer (term())

vocab(ref)

@spec vocab(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}