Planck.AI.Models.LlamaCpp (Planck.AI v0.1.0)

Copy Markdown View Source

Factory for llama.cpp models via its OpenAI-compatible HTTP server.

Unlike other catalog modules, this provides a factory function rather than a static list because the available model depends on what the user has loaded into their local llama.cpp server.

Examples

iex> Planck.AI.Models.LlamaCpp.model("llama3.2")
%Planck.AI.Model{provider: :llama_cpp, base_url: "http://localhost:8080", ...}

iex> Planck.AI.Models.LlamaCpp.model("mistral", base_url: "http://10.0.0.5:8080", context_window: 32_768)

Summary

Functions

model(id, opts \\ [])

Builds a Planck.AI.Model for a llama.cpp-hosted model.

Functions

model(id, opts \\ [])

@spec model(
  String.t(),
  keyword()
) :: Planck.AI.Model.t()

Builds a Planck.AI.Model for a llama.cpp-hosted model.

Options

  • :base_url — base URL of the llama.cpp server. Defaults to http://localhost:8080.
  • :context_window — context window size. Defaults to 4096.
  • :max_tokens — max tokens to generate. Defaults to 2048.
  • :supports_thinking — whether the model supports thinking blocks. Defaults to false.
  • :input_types — list of supported input modalities. Defaults to [:text].
  • :default_opts — inference parameters applied on every call unless overridden by the caller (e.g. [temperature: 1.0, top_p: 0.95, top_k: 40, min_p: 0.01]). Defaults to [].