Webhooks Explained: Signed Payloads and Retries

You've just integrated a third-party payment provider, and it promises to notify your app when a payment succeeds via a webhook. But how do you know the request actually came from them and not an attacker? And what happens if your server is briefly down when the webhook fires? Without proper safeguards, you risk processing fake events or losing critical updates. This article explains how to secure webhooks with signed payloads and implement robust retry mechanisms.

What Is a Webhook?

A webhook is a user-defined HTTP callback. When an event occurs in a source system (e.g., a payment is completed), it sends an HTTP POST request to a URL you provide. Unlike polling, where you repeatedly ask for updates, webhooks push data to you in near real-time. They are widely used for event notifications, such as CI/CD triggers, chat messages, or payment status changes.

Why Secure Webhooks with Signatures?

Webhook endpoints are publicly accessible, so anyone can send a POST request to them. Without verification, an attacker could forge events, leading to unauthorized actions like marking an order as paid. Signing payloads with a shared secret allows you to verify authenticity and integrity.

How Signed Payloads Work

The sender and receiver share a secret key. For each webhook, the sender computes a cryptographic signature (usually an HMAC) over the request body and includes it in a header. The receiver recomputes the signature using the same secret and compares it. If they match, the request is genuine.

Choosing the Right Algorithm

HMAC-SHA256 is the de facto standard. It's fast, secure, and available in all major languages. Avoid MD5 or SHA1, which are considered weak.

Step-by-Step: Verifying a Signed Payload

1. Extract the signature from the header (e.g., X-Signature).
2. Get the raw request body as a string or byte array. Do not parse it first; any transformation will change the signature.
3. Compute the HMAC using your secret and the raw body.
4. Compare the computed signature with the received one using a constant-time comparison to prevent timing attacks.

Here's an example in Node.js using Express:

const crypto = require('crypto');
const express = require('express');
const app = express();

// Use raw body parser for webhook route
app.post('/webhook', express.raw({ type: 'application/json' }), (req, res) => {
  const signature = req.headers['x-signature'];
  const secret = process.env.WEBHOOK_SECRET;

  const hmac = crypto.createHmac('sha256', secret);
  const digest = hmac.update(req.body).digest('hex');

  if (!crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(digest))) {
    return res.status(401).send('Invalid signature');
  }

  // Process the event
  const event = JSON.parse(req.body);
  // ... handle event

  res.status(200).send('OK');
});

Common Pitfall: Using a JSON body parser before verification. Express's express.json() parses the body, so the raw bytes are lost. Always use express.raw() for webhook routes or capture the raw body separately.

Designing a Retry Strategy

Even with signatures, webhooks can fail: your server might be down, or the request might time out. A reliable sender retries failed deliveries. As a receiver, you should also handle retries gracefully.

Sender-Side Retries

If you're building the sender, implement exponential backoff with jitter. For example, retry after 1 minute, then 5 minutes, then 30 minutes, up to a maximum number of attempts (e.g., 5). Include a unique event ID in the payload so the receiver can deduplicate.

Receiver-Side Idempotency

Because retries can cause duplicate deliveries, your endpoint must be idempotent. Use the event ID to track processed events. Store it in a database with a unique constraint, or use a cache like Redis with a TTL. If the event ID has been seen, return a 200 OK immediately without reprocessing.

Example using a database unique index:

async function handleWebhook(event) {
  try {
    await db.query('INSERT INTO processed_events (event_id) VALUES ($1)', [event.id]);
  } catch (err) {
    if (err.code === '23505') { // unique violation
      return; // already processed
    }
    throw err;
  }
  // process event
}

Best Practices for Webhook Endpoints

• Respond quickly: Return a 2xx status within a few seconds. Offload heavy processing to a background job.
• Validate timestamp: Include a timestamp in the signature to prevent replay attacks. Reject requests older than a few minutes.
• Use HTTPS: Always use TLS to protect the payload in transit.
• Log everything: Log incoming webhooks, including headers and body, for debugging.
• Monitor failures: Set up alerts for repeated failures or signature mismatches.

Comparison: Signed vs Unsigned Webhooks

FAQ

How do I verify a webhook signature in Python?

Use the hmac module with SHA256. Compare the computed digest to the header using hmac.compare_digest for constant-time comparison.

What should I do if my webhook endpoint receives a duplicate event?

Check the event ID against a store of processed IDs. If it's a duplicate, return a 200 OK and skip processing. This ensures idempotency.

How many retries should a webhook sender perform?

There's no fixed number, but 3–5 retries with exponential backoff (e.g., 1min, 5min, 30min) is common. After that, mark the delivery as failed and alert.

Ready to test your webhook endpoint? Use our JSON Formatter to pretty-print and validate payloads during development.