Agentic Systemsadvanced13 min · 9 steps
Evaluating Agentic Systems
Score the whole trajectory, not just the answer: efficiency, task and action success, state-based checks, judges — and an eval harness hardened against agents that cheat.
tool calldataerrorsignalrequestresponse
ProblemStep 1 of 9
Two agents, one correct answer, very different runs
Agent A takes three tool calls. Agent B loops through the same search five times, calls a write tool without approval, and eventually lands on the same final answer. A response-level eval scores them identically.
RAG evaluation asks is this answer grounded? Agent evaluation has to ask about reliability, efficiency and safety — so the unit of evaluation is the trajectory, every step.
| RAG app | Agentic app | |
|---|---|---|
| Unit of eval | Single response | The trajectory (all steps) |
| Success | Groundedness / faithfulness | Task completion, sound logic |
| Complexity | Text similarity | Tool-state validation |
Lesson 10 of 15 in Agentic Systems