Backend for AIintermediate11 min · 10 steps
Retries with Backoff
Re-attempt failed calls after growing, randomised delays so transient failures heal without hammering a struggling dependency or synchronising thousands of clients into a retry storm.
heavy workerrorrequesttoken
ProblemStep 1 of 10
Everyone retries at the same moment
The model provider has a 30-second hiccup. A thousand clients see errors and retry immediately, then again after exactly 1 s, then 2 s. Every wave lands at the same instant, on a service that is trying to recover.
That is a retry storm: the retries themselves keep the dependency overloaded long after the original problem is gone.
Watch outRetries without backoff and jitter turn a short blip into a long outage.
Lesson 16 of 20 in Backend for AI