Captures the structured sequence of messages and tool calls produced during an LLMChain run for inspection, serialization, and comparison.

A trajectory provides a first-class API for extracting the decision-making path from a chain run — which tools were called, in what order, with what arguments — enabling golden-file testing, logging, and correctness assertions for agent workflows.

Why trajectories matter

When building agent systems, the final answer is only part of the story. Two agents can produce the same answer through very different reasoning paths — one might make a single efficient tool call while another makes five redundant ones. Trajectories let you evaluate the process, not just the outcome.

This is especially important for:

• Regression testing — catch when a prompt change causes the agent to take a different (possibly worse) path even if the final answer is correct
• Cost control — detect unnecessary tool calls that waste tokens and time
• Safety — verify that dangerous tools were NOT called
• Debugging — understand exactly what the agent did and why

Usage

trajectory = Trajectory.from_chain(chain)

# Serialize for logging or golden-file comparison
map = Trajectory.to_map(trajectory)

# Deserialize back from stored map
trajectory = Trajectory.from_map(map)

# Compare against expected tool call sequence
Trajectory.matches?(trajectory, [
  %{name: "search", arguments: %{"query" => "weather"}},
  %{name: "get_forecast", arguments: nil}
])

# Filter tool calls by name
Trajectory.calls_by_name(trajectory, "search")

# Group tool calls by conversation turn
Trajectory.calls_by_turn(trajectory)

Metadata

Each trajectory captures metadata about the chain run including the model name and LLM module. You can also add custom metadata:

trajectory = Trajectory.from_chain(chain)
trajectory.metadata
#=> %{model: "gpt-4", llm_module: LangChain.ChatModels.ChatOpenAI}

Evaluation patterns

Golden-file testing

Save a known-good trajectory and compare future runs against it:

# Save the golden file
golden = chain |> Trajectory.from_chain() |> Trajectory.to_map()
File.write!("test/fixtures/weather_agent.json", Jason.encode!(golden))

# In your test
golden = "test/fixtures/weather_agent.json" |> File.read!() |> Jason.decode!()
expected = Trajectory.from_map(golden)
actual = Trajectory.from_chain(chain)
assert Trajectory.matches?(actual, expected)

Verifying tools were NOT called

Use refute with superset mode to ensure dangerous tools weren't invoked:

# Using Trajectory.Assertions
use LangChain.Trajectory.Assertions

refute_trajectory trajectory, [
  %{name: "delete_all", arguments: nil}
], mode: :superset

Flexible matching

When you care about which tools were called but not exact arguments:

Trajectory.matches?(trajectory, [
  %{name: "search", arguments: nil},
  %{name: "summarize", arguments: nil}
])

When you care that certain tools were called but allow extra calls:

Trajectory.matches?(trajectory, [
  %{name: "search", arguments: nil}
], mode: :superset)

Comparison modes

The matches?/3 function supports three modes via the :mode option:

• :strict (default) — same tool calls in the same order and count
• :unordered — same tool calls in any order, same count
• :superset — actual contains at least all expected calls

And two argument comparison modes via the :args option:

• :exact (default) — arguments must match exactly
• :subset — expected arguments must be a subset of actual arguments

External references

For more on trajectory-based evaluation of agent systems, see:

• LangSmith Trajectory Evaluation — trajectory-level evaluators for scoring agent behavior
• AgentEvals — reference implementations of trajectory matching algorithms

Arguments use string keys

Tool call arguments come from JSON decoding and use string keys (e.g. %{"city" => "Paris"} not %{city: "Paris"}). Expected arguments in matches?/3 should use string keys as well.