Deployment & Deliveryintermediate10 min · 10 steps
Prompt Versioning
Treat prompts like code: immutable versions with author, reason and eval results, labels that point at what's live, and the version on every trace.
VPN thumbs-down11%
dataerror
metrics are illustrativeProblemStep 1 of 10
Answers got worse, and nobody changed any code
On Wednesday, thumbs-down on VPN answers jumps from 6% to 11% (illustrative). No deploy happened. The code is identical. The model is the same snapshot.
Someone improved the system prompt in a web console on Tuesday afternoon. There is no record of the old text, no eval result, and nothing on the traces to say which prompt answered which question.
Watch outAn unversioned prompt edit is a production deploy with no diff, no review and no rollback.
Lesson 16 of 20 in Deployment & Delivery