Testing Agents — Tools, Flows and Human Review
A small change to Tripmate's prompt makes it write dates as "16/10/2026". The prompt reads fine, the demo looks fine, and a week later bookings start failing because the railway tool expects "2026-10-16". An agent has many moving parts, and each kind of part needs its own kind of test.
The rule
Test tools like code, flows by outcome, and quality with people.
Three layers, from cheapest to most expensive:
- Unit tests for individual tools — tools are ordinary functions, so test them without any model at all. Feed fixed inputs, check exact outputs and errors. Fast, repeatable, and they catch most plain bugs.
- Integration tests for flows — run the whole agent on a scripted request, with outside services replaced by fakes that return known data. Because model wording varies from run to run, check outcomes and properties, not exact text: the right tool was called, with valid arguments, the booking stayed within budget, the agent asked before paying.
- Human-in-the-loop evaluation — people review a sample of real or realistic conversations for things code cannot judge well: was the advice sensible, was the tone right, did it misunderstand the user? Their verdicts become new test cases.
Some teams add a self-critique step inside the agent — a second model call that checks a draft answer against the goal before it is sent. It helps, but it is not a replacement for tests, because the checker can share the same blind spots.
def test_search_trains_rejects_bad_date():
result = search_trains(frm="PUNE", to="JP", date="16/10/2026")
assert result["error"] == "date must be YYYY-MM-DD"
def test_booking_flow_stays_in_budget():
run = run_agent("Cheapest train Pune to Jaipur on 16 Oct, max ₹2,500", fakes=TIMETABLE_FAKES)
assert run.called("search_trains")
assert run.booking_total() <= 2500
assert run.asked_user_before("book_ticket")The shortcut
For each thing that can go wrong, ask: "Can code check this with a fixed answer?" Yes → unit test. "Can code check it as a property of the run?" Yes → integration test. Only a person can judge it? → human review, then turn the finding into a new automated test if you can.
Unit layer
search_trainsgiven "16/10/2026" should return a clear error. A test proves it does — the tool is fine.
The trap is testing flows by exact wording. The model may phrase a correct answer differently each run, so exact-match tests fail at random and teach the team to ignore them.
Drills
Answer fast — trust the rule.
How should you test the get_weather tool's handling of a missing city?
Why do flow tests check properties rather than exact reply text?
Which check is best left to human reviewers?
Why is a self-critique step not a full substitute for tests?
Remember
- Test tools like code, flows by outcome, and quality with people.
- Shortcut: fixed answer → unit test; property of the run → integration test; judgement → human review.
- Trap: exact-wording tests on model output, which fail at random.