The Webhook Failure Modes Nobody Warns You About

Webhooks get treated like a solved problem — add an endpoint, handle the event, done. Then one arrives twice, one arrives out of order, one never arrives at all, and whatever you built quietly goes wrong in a way nobody notices until a customer complains.

Here are the failure modes that actually show up, and what handles each one.

Duplicate delivery

Stripe (and most webhook providers) explicitly do not guarantee exactly-once delivery — the same event can arrive twice. If your handler for "subscription created" sends a welcome email or grants access every time it fires, a duplicate delivery double-sends the email or double-grants something it shouldn't. Fix: check the event ID against ones you've already processed before acting on it, and make the whole handler idempotent — running it twice on the same event should produce the same end state as running it once.

Out-of-order delivery

Events can arrive in a different order than they happened. If subscription.updated (downgrade) arrives before subscription.created (the original signup) because of a network hiccup, and your handler assumes creation always comes first, you can end up processing state changes in the wrong order. Fix: don't assume order — pull the current state of the object from the provider's API inside the handler rather than trusting the payload's implied sequence, especially for anything that changes access level.

Failed delivery, silent gap

If your endpoint is down for even a few minutes during a deploy, events fire during that window and, depending on the provider's retry policy, may or may not come back. This is exactly how a subscription can quietly desync from what your database thinks is true, and nobody notices until someone can't access something they paid for, or can access something they cancelled. Fix: don't treat webhooks as your only sync mechanism — run a periodic reconciliation job that pulls actual subscription state from the provider and corrects any drift, so a missed webhook self-heals instead of staying wrong forever.

Signature verification skipped "for now"

Every webhook endpoint should verify the payload's signature against your provider's signing secret before trusting anything in it. Skipping this "temporarily" during development and forgetting to add it back is one of the most common real holes — it means anyone who finds your endpoint URL can POST a fake event and trigger whatever that handler does, including granting paid access for free.

Treating the webhook handler as the only path

The handler should update your database and nothing more time-sensitive than that. Anything user-facing that depends on the webhook having fired recently (like showing "processing your payment...") should also have a reasonable timeout and fallback state, because a slow or delayed webhook shouldn't leave a user staring at a spinner with no explanation.

None of these are exotic edge cases. They're the default behavior of every webhook system, and the gap between "webhooks work" and "webhooks work correctly under real conditions" is exactly these five things.