Retries are capped and backed off, and every outbound call times out
Three layers each retrying three times is 27 requests for one failure. AWS's October 2025 postmortem calls the result "congestive collapse," where engineers throttled traffic by hand. Node's fetch has no useful default timeout, so a hung upstream is a function you pay for by the second.
Where it bit
Not in ours, by luck more than design. It is the retry storm, the one that multiplies, and the most expensive of the five in this group.
The practice
Retry at one layer only, capped exponential backoff with jitter, honor Retry-After. AbortSignal.timeout(5000) on every fetch, statement_timeout on Postgres, explicit maxDuration on functions. A circuit breaker for anything that can go down for an hour.
Check it
grep -rn "fetch(" src/ | grep -vc "signal" # fetches with no abort signal
grep -rnE "retry|retries" src/ | grep -viE "backoff|jitter|max"Get this check as a script you can run tonight
What to do with this
If you run a business on something AI built and the checks came back with more than you expected, that is worth a conversation.
We do a free 30-minute Health Check for service businesses that want to know exactly where their biggest leaks are. No slide deck. No pitch. We ask questions, find the gaps, and tell you what we see. If there is no obvious fix, we will tell you that too.
Blinkz finds what is broken in how a business runs, then fixes it. AI only where it earns its place.