ExUnit helpers for testing agents built with Alloy.
use Alloy.Testing imports convenience functions that eliminate
boilerplate when writing tests for your agent. All helpers use
Alloy.Provider.Test under the hood — no HTTP calls are made.
Example
defmodule MyAgent.Test do
use ExUnit.Case, async: true
use Alloy.Testing
test "agent greets the user" do
result = run_with_responses("Hello", [
text_response("Hi there!")
])
assert result.status == :completed
assert last_text(result) == "Hi there!"
end
test "agent uses the weather tool" do
result = run_with_responses("What's the weather?", [
tool_response("get_weather", %{location: "Sydney"}),
text_response("It's 22°C in Sydney.")
])
assert_tool_called(result, "get_weather")
assert last_text(result) =~ "Sydney"
end
endSummary
Functions
Assert that a tool was called during the conversation.
Assert that a tool was called with input matching a partial map.
Build a scripted error response for the test provider.
Extract the text content from the last assistant message in a result.
Assert that a tool was NOT called during the conversation.
Run the agent turn loop with scripted provider responses.
Build a scripted text response for the test provider.
Extract all tool call blocks from the conversation.
Build a scripted tool use response for the test provider.
Functions
Assert that a tool was called during the conversation.
Optionally match against the tool's input with a partial map.
Examples
assert_tool_called(result, "bash")
assert_tool_called(result, "read", %{"file_path" => "mix.exs"})Assert that a tool was called with input matching a partial map.
Build a scripted error response for the test provider.
Shortcut for Alloy.Provider.Test.error_response/1.
@spec last_text(Alloy.Agent.State.t() | map()) :: String.t() | nil
Extract the text content from the last assistant message in a result.
Returns nil if there are no assistant messages.
Assert that a tool was NOT called during the conversation.
@spec run_with_responses(String.t(), [term()], keyword()) :: Alloy.Agent.State.t()
Run the agent turn loop with scripted provider responses.
Takes a user prompt and a list of scripted responses (built with
text_response/1, tool_response/2, or error_response/1).
Returns the final Alloy.Agent.State struct.
Options
:tools- list of tool modules (default:[Alloy.Test.EchoTool]):system_prompt- system prompt string:max_turns- maximum turns (default: 10):middleware- list of middleware modules:working_directory- working directory for tools
Build a scripted text response for the test provider.
Shortcut for Alloy.Provider.Test.text_response/1.
@spec tool_calls(Alloy.Agent.State.t() | map()) :: [map()]
Extract all tool call blocks from the conversation.
Returns a list of maps with :name, :id, and :input keys.
Build a scripted tool use response for the test provider.
Takes a tool name and input map, generates a tool call block with a unique ID.