View Source ExOpenAI.FineTuning (ex_openai.ex v1.7.0)

Modules for interacting with the fine_tuning group of OpenAI APIs

API Reference: https://platform.openai.com/docs/api-reference/fine_tuning

Summary

Functions

Immediately cancel a fine-tune job.

Creates a fine-tuning job which begins the process of creating a new model from a given dataset.

Get status updates for a fine-tuning job.

List your organization's fine-tuning jobs

Functions

Link to this function

cancel_fine_tuning_job(fine_tuning_job_id, opts \\ [])

View Source
@spec cancel_fine_tuning_job(String.t(),
  base_url: String.t(),
  openai_organization_key: String.t(),
  openai_api_key: String.t()
) :: {:ok, ExOpenAI.Components.FineTuningJob.t()} | {:error, any()}

Immediately cancel a fine-tune job.

Endpoint: https://api.openai.com/v1/fine_tuning/jobs/{fine_tuning_job_id}/cancel

Method: POST

Docs: https://platform.openai.com/docs/api-reference/fine_tuning


Required Arguments:

  • fine_tuning_job_id

Example: ft-AF1WoRqd3aJAHsqc9NY7iL8F

Optional Arguments:

  • openai_api_key: OpenAI API key to pass directly. If this is specified, it will override the api_key config value.

  • openai_organization_key: OpenAI API key to pass directly. If this is specified, it will override the organization_key config value.

  • base_url: Which API endpoint to use as base, defaults to https://api.openai.com/v1

Link to this function

create_fine_tuning_job(model, training_file, opts \\ [])

View Source
@spec create_fine_tuning_job(
  (:"gpt-3.5-turbo" | :"davinci-002" | :"babbage-002") | String.t(),
  String.t(),
  base_url: String.t(),
  openai_organization_key: String.t(),
  openai_api_key: String.t(),
  validation_file: String.t(),
  suffix: String.t(),
  seed: integer(),
  integrations: [
    %{
      type: :wandb,
      wandb: %{
        entity: String.t(),
        name: String.t(),
        project: String.t(),
        tags: [String.t()]
      }
    }
  ],
  hyperparameters: %{
    batch_size: integer() | :auto,
    learning_rate_multiplier: float() | :auto,
    n_epochs: integer() | :auto
  },
  stream_to: (... -> any()) | pid()
) :: {:ok, ExOpenAI.Components.FineTuningJob.t()} | {:error, any()}

Creates a fine-tuning job which begins the process of creating a new model from a given dataset.

Response includes details of the enqueued job including job status and the name of the fine-tuned models once complete.

Learn more about fine-tuning

Endpoint: https://api.openai.com/v1/fine_tuning/jobs

Method: POST

Docs: https://platform.openai.com/docs/api-reference/fine_tuning


Required Arguments:

  • model: The name of the model to fine-tune. You can select one of the supported models.

  • training_file: The ID of an uploaded file that contains training data.

See upload file for how to upload a file.

Your dataset must be formatted as a JSONL file. Additionally, you must upload your file with the purpose fine-tune.

The contents of the file should differ depending on if the model uses the chat or completions format.

See the fine-tuning guide for more details.

Example: file-abc123

Optional Arguments:

  • stream_to: PID or function of where to stream content to

  • hyperparameters: The hyperparameters used for the fine-tuning job.

  • integrations: A list of integrations to enable for your fine-tuning job.

  • seed: The seed controls the reproducibility of the job. Passing in the same seed and job parameters should produce the same results, but may differ in rare cases. If a seed is not specified, one will be generated for you.

Example: 42

  • suffix: A string of up to 18 characters that will be added to your fine-tuned model name.

For example, a suffix of "custom-model-name" would produce a model name like ft:gpt-3.5-turbo:openai:custom-model-name:7p4lURel.

  • validation_file: The ID of an uploaded file that contains validation data.

If you provide this file, the data is used to generate validation metrics periodically during fine-tuning. These metrics can be viewed in the fine-tuning results file. The same data should not be present in both train and validation files.

Your dataset must be formatted as a JSONL file. You must upload your file with the purpose fine-tune.

See the fine-tuning guide for more details.

Example: "file-abc123"

  • openai_api_key: OpenAI API key to pass directly. If this is specified, it will override the api_key config value.

  • openai_organization_key: OpenAI API key to pass directly. If this is specified, it will override the organization_key config value.

  • base_url: Which API endpoint to use as base, defaults to https://api.openai.com/v1

Link to this function

list_fine_tuning_events(fine_tuning_job_id, opts \\ [])

View Source
@spec list_fine_tuning_events(String.t(),
  base_url: String.t(),
  openai_organization_key: String.t(),
  openai_api_key: String.t(),
  limit: integer(),
  after: String.t(),
  stream_to: (... -> any()) | pid()
) ::
  {:ok, ExOpenAI.Components.ListFineTuningJobEventsResponse.t()}
  | {:error, any()}

Get status updates for a fine-tuning job.

Endpoint: https://api.openai.com/v1/fine_tuning/jobs/{fine_tuning_job_id}/events

Method: GET

Docs: https://platform.openai.com/docs/api-reference/fine_tuning


Required Arguments:

  • fine_tuning_job_id

Example: ft-AF1WoRqd3aJAHsqc9NY7iL8F

Optional Arguments:

  • stream_to: PID or function of where to stream content to

  • after

  • limit

  • openai_api_key: OpenAI API key to pass directly. If this is specified, it will override the api_key config value.

  • openai_organization_key: OpenAI API key to pass directly. If this is specified, it will override the organization_key config value.

  • base_url: Which API endpoint to use as base, defaults to https://api.openai.com/v1

Link to this function

list_fine_tuning_job_checkpoints(fine_tuning_job_id, opts \\ [])

View Source
@spec list_fine_tuning_job_checkpoints(String.t(),
  base_url: String.t(),
  openai_organization_key: String.t(),
  openai_api_key: String.t(),
  limit: integer(),
  after: String.t(),
  stream_to: (... -> any()) | pid()
) ::
  {:ok, ExOpenAI.Components.ListFineTuningJobCheckpointsResponse.t()}
  | {:error, any()}

List checkpoints for a fine-tuning job.

Endpoint: https://api.openai.com/v1/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints

Method: GET

Docs: https://platform.openai.com/docs/api-reference/fine_tuning


Required Arguments:

  • fine_tuning_job_id

Example: ft-AF1WoRqd3aJAHsqc9NY7iL8F

Optional Arguments:

  • stream_to: PID or function of where to stream content to

  • after

  • limit

  • openai_api_key: OpenAI API key to pass directly. If this is specified, it will override the api_key config value.

  • openai_organization_key: OpenAI API key to pass directly. If this is specified, it will override the organization_key config value.

  • base_url: Which API endpoint to use as base, defaults to https://api.openai.com/v1

Link to this function

list_paginated_fine_tuning_jobs(opts \\ [])

View Source
@spec list_paginated_fine_tuning_jobs(
  base_url: String.t(),
  openai_organization_key: String.t(),
  openai_api_key: String.t(),
  limit: integer(),
  after: String.t(),
  stream_to: (... -> any()) | pid()
) ::
  {:ok, ExOpenAI.Components.ListPaginatedFineTuningJobsResponse.t()}
  | {:error, any()}

List your organization's fine-tuning jobs

Endpoint: https://api.openai.com/v1/fine_tuning/jobs

Method: GET

Docs: https://platform.openai.com/docs/api-reference/fine_tuning


Required Arguments:

Optional Arguments:

  • stream_to: PID or function of where to stream content to

  • after

  • limit

  • openai_api_key: OpenAI API key to pass directly. If this is specified, it will override the api_key config value.

  • openai_organization_key: OpenAI API key to pass directly. If this is specified, it will override the organization_key config value.

  • base_url: Which API endpoint to use as base, defaults to https://api.openai.com/v1

Link to this function

retrieve_fine_tuning_job(fine_tuning_job_id, opts \\ [])

View Source
@spec retrieve_fine_tuning_job(String.t(),
  base_url: String.t(),
  openai_organization_key: String.t(),
  openai_api_key: String.t()
) :: {:ok, ExOpenAI.Components.FineTuningJob.t()} | {:error, any()}

Get info about a fine-tuning job.

Learn more about fine-tuning

Endpoint: https://api.openai.com/v1/fine_tuning/jobs/{fine_tuning_job_id}

Method: GET

Docs: https://platform.openai.com/docs/api-reference/fine_tuning


Required Arguments:

  • fine_tuning_job_id

Example: ft-AF1WoRqd3aJAHsqc9NY7iL8F

Optional Arguments:

  • openai_api_key: OpenAI API key to pass directly. If this is specified, it will override the api_key config value.

  • openai_organization_key: OpenAI API key to pass directly. If this is specified, it will override the organization_key config value.

  • base_url: Which API endpoint to use as base, defaults to https://api.openai.com/v1