GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpec (google_api_ai_platform v0.27.0)

View Source

Configuration for Speculative Decoding.

Attributes

  • draftModelSpeculation (type: GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecDraftModelSpeculation.t, default: nil) - draft model speculation.
  • ngramSpeculation (type: GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation.t, default: nil) - N-Gram speculation.
  • speculativeTokenCount (type: integer(), default: nil) - The number of speculative tokens to generate at each step.

Summary

Types

t()

Functions

decode(value, options)

Unwrap a decoded JSON object into its complex fields.

Types

t()

@type t() ::
  %GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpec{
    draftModelSpeculation:
      GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecDraftModelSpeculation.t()
      | nil,
    ngramSpeculation:
      GoogleApi.AIPlatform.V1.Model.GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation.t()
      | nil,
    speculativeTokenCount: integer() | nil
  }

Functions

decode(value, options)

@spec decode(struct(), keyword()) :: struct()

Unwrap a decoded JSON object into its complex fields.