Logging, Tracing and Metrics
Asha complains: "Tripmate took two minutes and then said no trains exist." In a normal program you would open the logs and see the error. In an agent, the answer is hidden in a chain of model calls and tool calls that happened once and will never repeat exactly. If you did not record that chain, you cannot explain it.
The rule
Record every step as structured data, and watch a few numbers that matter.
Structured logging means each event is recorded as fields, not a free-text sentence, so it can be searched and counted:
{"run_id": "r-8841", "step": 3, "type": "tool_call", "tool": "search_trains",
"args": {"from": "PUNE", "to": "JP", "date": "2026-10-16"},
"status": "error", "error": "timeout", "ms": 30000, "input_tokens": 6120}A trace ties all the events of one run together: every model call and tool call becomes a timed span, nested under the request that caused it. Reading a trace is like replaying the run: what the model saw, what it asked for, what came back, how long each step took, and what it cost.
Metrics summarise many runs. A small, steady set covers most needs:
- Task success rate — share of runs that achieved the user's goal.
- Steps per task — rising numbers often mean the agent is wandering.
- Latency — time to the final answer.
- Cost per task — tokens and money, ideally per successful task.
- Tool error rate — per tool, so a failing integration stands out.
- Escalation rate — how often the agent had to hand over to a person.
Logs hold user data, so apply the same care as anywhere else: redact personal details such as phone numbers, and keep logs only as long as you need them.
The shortcut
When something goes wrong, go metric → trace → step. A metric tells you that something changed, a trace of one bad run tells you where, and the step inside it tells you why.
Metric
The dashboard shows
search_trainserrors jumped from 1% to 30% this morning.
The trap is logging only the final answer. Without the steps in between, every failure looks like "the model was wrong", and you cannot tell a bad prompt from a broken tool.
Drills
Answer fast — trust the rule.
200 Tripmate runs; 170 met the user's goal. What is the task success rate?
What is a trace in an agent system?
Why log {"tool": "search_trains", "status": "error"} rather than "search failed again, ugh"?
Steps per task rose from 6 to 14 after a prompt change, with no rise in success. What does it most likely signal?
Remember
- Log every step as structured fields, tie them into traces, and track a few key metrics.
- Shortcut: metric → trace → step: that, where, why.
- Trap: logging only final answers — and logging personal data without redaction.