Evaluation & Observabilityintermediate13 min · 9 steps
Eval Tooling: LangWatch and Langfuse
What an eval platform gives you (traces, datasets, experiments, online evaluators) and how to keep judge costs and latency in check.
dataresponserequest
ProblemStep 1 of 9
Without traces, you can't do evals
A trace records everything the AI did for one request: system prompt, user message, tool calls, tool results and the reply, plus model parameters, tokens, latency and cost.
This one shows two bugs that the final reply alone would hide. The tool was called with bathroom_connected=None, so the user's constraint was dropped, and the SMS reply used markdown.
TipMinimum: input, output, timestamp and a unique ID. Better: prompts, tool calls, tokens, latency, cost. Best: session history, errors, model version and feature flags.
What changed
- +Instrumentation
- +One full trace
Lesson 5 of 5 in Evaluation & Observability