Skip to content
anythingai.space
Backend for AIintermediate11 min · 10 steps

Retries with Backoff

Re-attempt failed calls after growing, randomised delays so transient failures heal without hammering a struggling dependency or synchronising thousands of clients into a retry storm.

retry, retry, retryFIXED RETRY DELAYS: EVERY CLIENT IN LOCKSTEPall clientsprovider loadspikespikespikespikeseconds →1,000 clientsModel APIprimary regionoverloaded
heavy workerrorrequesttoken
1 / 10
ProblemStep 1 of 10

Everyone retries at the same moment

The model provider has a 30-second hiccup. A thousand clients see errors and retry immediately, then again after exactly 1 s, then 2 s. Every wave lands at the same instant, on a service that is trying to recover.

That is a retry storm: the retries themselves keep the dependency overloaded long after the original problem is gone.

Watch outRetries without backoff and jitter turn a short blip into a long outage.

Lesson 16 of 20 in Backend for AI