starlet - v1.0.1

starlet

A unified, provider-agnostic interface for LLM APIs.

Quick Start

import starlet
import starlet/ollama

let client = ollama.new("http://localhost:11434")

let chat =
  starlet.chat(client, "qwen3:0.6b")
  |> starlet.system("You are a helpful assistant.")
  |> starlet.user("Hello!")

case starlet.send(chat) {
  Ok(#(new_chat, turn)) -> starlet.text(turn)
  Error(err) -> // handle error
}

Typestate

The Chat type uses phantom types to enforce correct usage at compile time:

Error Handling

import starlet.{Transport, Http, Decode, Provider}

case starlet.send(chat) {
  Ok(#(chat, turn)) -> // success
  Error(Transport(msg)) -> // network error
  Error(Http(status, body)) -> // non-200 response
  Error(Decode(msg)) -> // JSON parse error
  Error(Provider(name, msg, raw)) -> // provider error
}

Types

Chat

</>

A conversation builder that accumulates messages and settings.

The type parameters track capabilities at compile time:

  • tools: Whether tool calling is enabled
  • format: Output format constraint (free text or JSON)
  • state: Whether the chat is ready to send (has at least one user message)
  • ext: Provider-specific extension data
pub type Chat(tools, format, state, ext) {
  Chat(
    client: Client(ext),
    model: String,
    system_prompt: option.Option(String),
    messages: List(@internal Message),
    tools: List(tool.Definition),
    temperature: option.Option(Float),
    max_tokens: option.Option(Int),
    ext: ext,
    json_schema: option.Option(json.Json),
    timeout_ms: Int,
  )
}

Constructors

Client

</>

An LLM provider client. Create one using a provider module like ollama.new().

The type parameter tracks the provider’s extension type, allowing provider-specific features (like reasoning effort) to flow through naturally.

pub type Client(ext) {
  Client(p: @internal ProviderConfig(ext), default_ext: ext)
}

Constructors 

  • Client(p: @internal ProviderConfig(ext), default_ext: ext)

StarletError

</>

Errors that can occur when interacting with LLM providers.

pub type StarletError {
  Transport(message: String)
  Http(status: Int, body: String)
  Decode(message: String)
  Provider(provider: String, message: String, raw: String)
  Tool(error: tool.ToolError)
  RateLimited(retry_after: option.Option(Int))
}

Constructors 

  • Transport(message: String)

    Network-level error (connection refused, timeout, etc.)

  • Http(status: Int, body: String)

    Non-200 HTTP response from the provider

  • Decode(message: String)

    Failed to parse the provider’s JSON response

  • Provider(provider: String, message: String, raw: String)

    Provider-specific error (model not found, rate limited, etc.)

  • Tool(error: tool.ToolError)

    Tool execution error

  • RateLimited(retry_after: option.Option(Int))

    Rate limited by the provider

Step

</>

The result of a single step in a tool-enabled conversation.

pub type Step(format, ext) {
  Done(
    chat: Chat(@internal ToolsOn, format, @internal Ready, ext),
    turn: Turn(@internal ToolsOn, format, ext),
  )
  ToolCall(
    chat: Chat(@internal ToolsOn, format, @internal Ready, ext),
    turn: Turn(@internal ToolsOn, format, ext),
    calls: List(tool.Call),
  )
}

Constructors 

  • Done(
  chat: Chat(@internal ToolsOn, format, @internal Ready, ext),
  turn: Turn(@internal ToolsOn, format, ext),
)

    Model responded with final text, no tool calls.

  • ToolCall(
  chat: Chat(@internal ToolsOn, format, @internal Ready, ext),
  turn: Turn(@internal ToolsOn, format, ext),
  calls: List(tool.Call),
)

    Model wants to call tools. Provide results to continue.

Turn

</>

A model response from a single turn of conversation.

pub type Turn(tools, format, ext) {
  Turn(text: String, tool_calls: List(tool.Call), ext: ext)
}

Constructors 

  • Turn(text: String, tool_calls: List(tool.Call), ext: ext)

Values

apply_tool_results

</> 
pub fn apply_tool_results(
  chat: Chat(@internal ToolsOn, format, @internal Ready, ext),
  calls: List(tool.Call),
  run: fn(tool.Call) -> Result(tool.ToolResult, tool.ToolError),
) -> Result(
  Chat(@internal ToolsOn, format, @internal Ready, ext),
  StarletError,
)

Run tools and apply their results in one step. The runner is called for each tool call; errors short-circuit.

assistant

</> 
pub fn assistant(
  chat: Chat(tools_state, format, @internal Ready, ext),
  text: String,
) -> Chat(tools_state, format, @internal Ready, ext)

Adds an assistant message to the chat history.

Useful for providing few-shot examples or resuming a conversation. Requires the chat to already have a user message.

chat

</> 
pub fn chat(
  client: Client(ext),
  model: String,
) -> Chat(
  @internal ToolsOff,
  @internal FreeText,
  @internal Empty,
  ext,
)

Creates a new chat with the given client and model name.

The chat inherits the provider’s extension type from the client, allowing provider-specific features to be configured.

let chat = starlet.chat(client, "qwen3:0.6b")

has_tool_calls

</> 
pub fn has_tool_calls(
  turn: Turn(@internal ToolsOn, format, ext),
) -> Bool

Check if a turn has any tool calls.

json

</> 
pub fn json(
  turn: Turn(tools_state, @internal JsonFormat, ext),
) -> String

Extracts the JSON content from a turn. Only available for JSON format turns.

max_tokens

</> 
pub fn max_tokens(
  chat: Chat(tools_state, format, state, ext),
  value: Int,
) -> Chat(tools_state, format, state, ext)

Sets the maximum number of tokens to generate in the response.

provider_name

</> 
pub fn provider_name(client: Client(ext)) -> String

Returns the name of the provider (e.g., “ollama”, “openai”).

send

</> 
pub fn send(
  chat: Chat(tools_state, format, @internal Ready, ext),
) -> Result(
  #(
    Chat(tools_state, format, @internal Ready, ext),
    Turn(tools_state, format, ext),
  ),
  StarletError,
)

Sends the chat to the LLM and returns the response.

Returns a tuple of the updated chat (with the assistant’s response appended to the history) and the turn containing the response text.

case starlet.send(chat) {
  Ok(#(new_chat, turn)) -> starlet.text(turn)
  Error(err) -> // handle error
}

step

</> 
pub fn step(
  chat: Chat(@internal ToolsOn, format, @internal Ready, ext),
) -> Result(Step(format, ext), StarletError)

Send a tools-enabled chat and categorize the response. Returns either Done (no tool calls) or ToolCall (tools requested).

system

</> 
pub fn system(
  chat: Chat(tools, format, @internal Empty, ext),
  text: String,
) -> Chat(tools, format, @internal Empty, ext)

Sets the system prompt for the chat.

Must be called before adding any user messages.

temperature

</> 
pub fn temperature(
  chat: Chat(tools_state, format, state, ext),
  value: Float,
) -> Chat(tools_state, format, state, ext)

Sets the sampling temperature (typically 0.0 to 2.0).

Lower values make output more deterministic, higher values more creative.

text

</> 
pub fn text(
  turn: Turn(tools_state, @internal FreeText, ext),
) -> String

Extracts the text content from a turn. Only available for free text format turns.

timeout

</> 
pub fn timeout(
  chat: Chat(tools_state, format, state, ext),
) -> Int

Returns the current timeout in milliseconds.

tool_calls

</> 
pub fn tool_calls(
  turn: Turn(@internal ToolsOn, format, ext),
) -> List(tool.Call)

Extract tool calls from a turn. Only available when tools are enabled.

tools

</> 
pub fn tools(
  chat: Chat(@internal ToolsOn, format, state, ext),
) -> List(tool.Definition)

Get the tool definitions from a tools-enabled chat.

user

</> 
pub fn user(
  chat: Chat(tools_state, format, state, ext),
  text: String,
) -> Chat(tools_state, format, @internal Ready, ext)

Adds a user message to the chat.

This transitions the chat to the Ready state, allowing it to be sent.

with_free_text

</> 
pub fn with_free_text(
  chat: Chat(tools, @internal JsonFormat, state, ext),
) -> Chat(tools, @internal FreeText, state, ext)

Disable JSON output, return to free text. Transitions JsonFormat → FreeText.

with_json_output

</> 
pub fn with_json_output(
  chat: Chat(tools, @internal FreeText, state, ext),
  output_schema: schema.Type,
) -> Chat(tools, @internal JsonFormat, state, ext)

Enable JSON output with a schema. Transitions FreeText → JsonFormat.

The model will be constrained to output valid JSON matching the schema. Use json(turn) to extract the JSON string from the response.

with_timeout

</> 
pub fn with_timeout(
  chat: Chat(tools_state, format, state, ext),
  timeout_ms: Int,
) -> Chat(tools_state, format, state, ext)

Sets the HTTP request timeout in milliseconds.

Default is 60,000ms (60 seconds). Increase for long-running requests.

starlet.chat(client, "gpt-4o")
|> starlet.with_timeout(120_000)  // 2 minutes
|> starlet.user("Solve this complex problem...")
|> starlet.send()

with_tool_results

</> 
pub fn with_tool_results(
  chat: Chat(@internal ToolsOn, format, @internal Ready, ext),
  results: List(tool.ToolResult),
) -> Chat(@internal ToolsOn, format, @internal Ready, ext)

Apply pre-computed tool results to the chat. Use when you’ve already run the tools yourself.

with_tools

</> 
pub fn with_tools(
  chat: Chat(@internal ToolsOff, format, state, ext),
  tool_defs: List(tool.Definition),
) -> Chat(@internal ToolsOn, format, state, ext)

Enable tools on a chat. Transitions ToolsOff → ToolsOn.

✨ Search Document