Deployment & Deliveryintermediate12 min · 9 steps
Eval Gates in CI
Turn eval suites into required checkpoints that block a change — code, prompt, model or index — from advancing when quality, safety, cost or latency slip.
tool-call acc Δ-4 pts
cost/turn Δ-22%
users exposed100%
requesterrorresponsedatasignal
metrics are illustrativeProblemStep 1 of 9
An eval nobody has to pass is just advice
The team has a good golden set. It runs in CI and posts a score… and the PR merges anyway, because the check isn't required and the deadline is today.
And the riskiest changes don't even trigger it: a model upgrade, a re-indexed knowledge base or a new tool never opens a pull request against the code.
Watch outA suite nobody must pass is advice. A required status check is a control.
Lesson 6 of 20 in Deployment & Delivery