Webhooks Explained: Signed Payloads and Retry Strategies

Why Webhooks Fail and How to Fix Them

You've just integrated a payment provider's webhook to update order statuses in your app. Everything works in testing, but in production, some events never arrive, or worse, you receive duplicates. Without proper safeguards, webhooks can become a source of silent failures and security holes.

Webhooks are HTTP callbacks triggered by events, but unlike typical API calls, they're push-based and asynchronous. This introduces two core challenges: verifying authenticity and ensuring delivery. In this guide, we'll tackle both with signed payloads and smart retries.

What Are Webhooks?

A webhook is a user-defined HTTP callback. When an event occurs in a source system (e.g., a new payment), it sends an HTTP POST request to a URL you provide, containing event data. This allows real-time reactions without polling.

Common use cases include payment notifications, CI/CD triggers, and chat integrations. But because webhooks traverse the public internet, they're vulnerable to spoofing and loss.

Securing Webhooks with Signed Payloads

How do you know a webhook truly comes from your provider? Signing the payload with a shared secret ensures integrity and authenticity.

How HMAC Signatures Work

The provider computes an HMAC (e.g., SHA-256) of the request body using a secret key, and sends the signature in a header (e.g., X-Signature). Your server recomputes the HMAC and compares it using a constant-time comparison to prevent timing attacks.

// Node.js example: verifying a webhook signature
const crypto = require('crypto');

function verifySignature(payload, signature, secret) {
  const hmac = crypto.createHmac('sha256', secret);
  hmac.update(payload);
  const digest = hmac.digest('hex');
  return crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(digest));
}

Best practices:

• Use a strong, unique secret per webhook endpoint.
• Include a timestamp in the signed payload to prevent replay attacks; reject requests older than a few minutes.
• Always use HTTPS to protect the payload in transit.

Implementing Reliable Retries

Even with signatures, network issues or downtime can cause webhook delivery to fail. Providers typically retry with exponential backoff, but you must also handle retries on your side.

Idempotency: The Key to Safe Retries

Retries can lead to duplicate processing. Use idempotency keys (often included in the payload or as a header) to ensure each event is processed exactly once. Store processed event IDs and ignore duplicates.

// Example: idempotent webhook handler
const processedEvents = new Set();

app.post('/webhook', (req, res) => {
  const eventId = req.body.id;
  if (processedEvents.has(eventId)) {
    return res.status(200).send('Already processed');
  }
  // process event...
  processedEvents.add(eventId);
  res.status(200).send('OK');
});

Retry Strategy on the Receiver Side

If your processing fails, respond with a 5xx status to trigger the provider's retry. For transient errors, you can also implement your own retry queue with backoff.

Monitoring and Debugging Webhooks

Log all incoming webhook requests with headers and payloads (redact sensitive data). Use a tool like the JSON Formatter to pretty-print payloads for easier inspection. Set up alerts for failed deliveries and track retry counts.

FAQ

What is a signed webhook payload?

A signed payload includes a cryptographic signature (usually HMAC) that verifies the request came from a trusted source and wasn't tampered with.

How many times should I retry a failed webhook?

Most providers retry 3–5 times with exponential backoff. As a receiver, respond with appropriate status codes and let the provider handle retries.

Can I use webhooks without HTTPS?

No—HTTPS is essential to encrypt the payload and prevent man-in-the-middle attacks. Never accept webhooks over plain HTTP.

Ready to inspect and debug webhook payloads? Try our free JSON Formatter to format and validate JSON data instantly.