Part 2 · 1 chapters · ~8 min

Retries, Backoff and Dead Letters

Which failures to retry (transient versus permanent), exponential backoff with full jitter, retry limits by count and time, retry budgets, poison messages, dead-letter queues with context, alerting and redrive tooling, and retrying safely when operations have side effects.

Retry well, then give up well

code

function shouldRetry(err) {
  if (err.status && err.status >= 400 && err.status < 500 && err.status !== 429) return false;   // permanent: bad input
  return true;                                                                                 // timeouts, 5xx, 429
}
const delay = (attempt, base = 2_000, cap = 300_000) => Math.random() * Math.min(cap, base * 2 ** attempt);   // full jitter

RETRIES WITH BACKOFF, THEN A DEAD LETTER

exponential backoff with jitter, a limit, and somewhere for failures to go

swipe the figure sideways, or tap expand for full screen

1/5

exponential backoff

Wait longer after each failure (2, 4, 8, 16 s...) so a struggling dependency gets room to recover.

double the wait each timegive the dependency room