Webhooks Explained: Signed Payloads and Retries

Backend2026-11-15TryQuickToolBox

Why Webhooks Break in Production

You've built a webhook endpoint, tested it locally, and everything works. Then production hits: a provider sends a duplicate event, your endpoint times out, or an attacker spoofs a request. Suddenly, your integration is unreliable and insecure.

Webhooks are simple in concept—HTTP callbacks triggered by events—but production-grade implementations require careful handling of security and reliability. This guide covers two critical aspects: signed payloads to verify authenticity and retry strategies to handle failures.

What Are Webhooks?

A webhook is a user-defined HTTP callback. When an event occurs in a source system (e.g., a payment succeeds), it sends an HTTP POST request to a URL you provide, containing data about the event. This push model is more efficient than polling.

But webhooks introduce challenges:

Let's tackle security first.

Securing Webhooks with Signed Payloads

Signed payloads use a shared secret to create a cryptographic signature of the request body. The sender includes this signature in a header (e.g., X-Signature). Your endpoint recomputes the signature and compares it. If they match, the request is authentic.

How HMAC Signing Works

Most providers use HMAC-SHA256. The signature is computed over the raw request body using the shared secret. Here's the flow:

  1. Provider and consumer share a secret key.
  2. Provider creates the payload, computes HMAC-SHA256(secret, payload), and sends it in a header.
  3. Consumer receives the request, reads the raw body, computes the same HMAC, and compares.

Important: Use the raw body exactly as received—parsing and re-serializing JSON can change the bytes and break the signature.

Implementing Signature Verification

Here's a Node.js example using Express:

const crypto = require('crypto');
const express = require('express');
const app = express();

// Use raw body parser for webhook routes
app.post('/webhook', express.raw({ type: 'application/json' }), (req, res) => {
  const signature = req.headers['x-signature'];
  const secret = process.env.WEBHOOK_SECRET;
  
  const expected = crypto
    .createHmac('sha256', secret)
    .update(req.body)
    .digest('hex');
  
  if (!crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))) {
    return res.status(401).send('Invalid signature');
  }
  
  // Process the event
  const event = JSON.parse(req.body);
  // ... handle event
  
  res.status(200).send('OK');
});

Always use a constant-time comparison like crypto.timingSafeEqual to prevent timing attacks.

Other Security Considerations

Implementing Reliable Retries

Even with a secure endpoint, network issues or temporary failures can cause webhook delivery to fail. A robust retry mechanism ensures events aren't lost.

Sender-Side Retry Strategy

If you're building a webhook sender, implement exponential backoff with jitter. For example:

Add random jitter to avoid thundering herd problems. Stop after a maximum number of attempts and alert.

Receiver-Side Best Practices

As a webhook consumer, you can't control the sender's retry policy, but you can make your endpoint resilient:

Handling Duplicates with Idempotency

Webhook providers often retry on timeouts, so you may receive the same event multiple times. Make your processing idempotent:

Example using a database unique index:

async function handleEvent(event) {
  try {
    await db.query('INSERT INTO processed_events (id) VALUES ($1)', [event.id]);
  } catch (err) {
    if (err.code === '23505') { // unique violation
      return; // already processed
    }
    throw err;
  }
  // process event
}

Comparison: Webhook Security Approaches

Approach Pros Cons
HMAC signature Strong integrity, widely supported Requires secret management
Basic Auth Simple Less secure, credentials in header
IP allowlist Easy to implement IPs can change, not robust

HMAC signatures are the de facto standard for webhook security.

Testing Webhooks Locally

Tools like ngrok or localtunnel expose your local server to the internet for testing. Always test signature verification and retry logic with simulated failures.

When inspecting JSON payloads from webhooks, formatting them can help debug. Try our JSON Formatter to pretty-print and validate webhook payloads.

FAQ

How do I verify a webhook signature?

Compute an HMAC of the raw request body using the shared secret, then compare it to the signature header using a constant-time comparison function.

What should I do if my webhook endpoint is down?

Most providers retry with exponential backoff. Ensure your endpoint returns a 2xx quickly and process asynchronously to minimize downtime impact.

Can I use webhooks without signatures?

It's risky. Without signatures, attackers can spoof requests. Always use signed payloads or at least basic authentication over HTTPS.

Conclusion

Webhooks are powerful but require attention to security and reliability. Implement HMAC signature verification to ensure authenticity, and design your system to handle retries and duplicates gracefully. With these practices, your webhook integrations will be robust and production-ready.