Webhooks Explained: Signed Payloads and Reliable Retries

Backend2026-11-17TryQuickToolBox

You've just integrated a third-party service that sends webhooks to your endpoint. Everything works in testing, but in production, you notice missed events, duplicate deliveries, and even forged requests. Webhooks are simple in concept but tricky to get right. This article explains how to secure webhooks with signed payloads and implement reliable retry logic so your integrations are robust and trustworthy.

What Are Webhooks and Why They Matter

Webhooks are user-defined HTTP callbacks triggered by events. Instead of polling an API for changes, you register a URL, and the service sends a POST request with event data whenever something happens. This push model reduces latency and server load, making it ideal for real-time integrations like payment notifications, CI/CD triggers, and chat messages.

However, webhooks introduce challenges: verifying authenticity, handling failures, and ensuring exactly-once processing. Let's tackle these one by one.

Securing Webhooks with Signed Payloads

Without verification, anyone can send fake requests to your webhook endpoint. Signed payloads solve this by including a cryptographic signature in the request headers, generated using a shared secret. You verify the signature to confirm the request came from the expected sender and wasn't tampered with.

How HMAC Signatures Work

Most services use HMAC (Hash-based Message Authentication Code) with SHA-256. The sender computes a hash of the request body (and sometimes a timestamp) using the secret, and includes it in a header like X-Signature. You recompute the hash on your side and compare.

Here's a typical verification flow in Node.js:

const crypto = require('crypto');

function verifySignature(payload, signature, secret) {
  const expected = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex');
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expected)
  );
}

Always use a timing-safe comparison to prevent timing attacks. Also, include a timestamp in the signed data and reject requests older than a few minutes to prevent replay attacks.

Best Practices for Signature Verification

Implementing Reliable Retries

Networks fail, servers restart, and endpoints time out. A robust webhook system must retry failed deliveries. But retries can cause duplicates, so you need idempotency.

Retry Strategies

Most providers use exponential backoff with jitter. For example, retry after 1 minute, then 5 minutes, then 30 minutes, up to a maximum number of attempts (e.g., 5). Include a unique event ID in the payload so the receiver can deduplicate.

On the receiving side, respond quickly with a 2xx status to acknowledge receipt, then process asynchronously. If processing fails, you can't rely on the sender to retry indefinitely—implement your own dead-letter queue or alerting.

Idempotency Keys

To handle duplicate deliveries, use an idempotency key (often the event ID) and store processed IDs for a reasonable period. Before processing, check if the ID has been seen; if so, return success without re-processing.

// Example: idempotency check in Express
app.post('/webhook', async (req, res) => {
  const eventId = req.body.id;
  if (await db.seenEvents.has(eventId)) {
    return res.status(200).send('Already processed');
  }
  await db.seenEvents.add(eventId);
  // process event asynchronously
  res.status(200).send('OK');
});

Comparison: Polling vs Webhooks

AspectPollingWebhooks
LatencyHigh (interval-based)Low (near real-time)
Server LoadHigh (frequent requests)Low (event-driven)
ComplexitySimple to implementRequires signature verification and retries
ReliabilityDepends on polling frequencyDepends on retry mechanism

Step-by-Step: Building a Secure Webhook Receiver

  1. Set up an endpoint that accepts POST requests over HTTPS.
  2. Verify the signature using the shared secret and reject invalid requests with 401.
  3. Check the timestamp to prevent replay attacks.
  4. Respond immediately with 200 OK after basic validation, then process asynchronously.
  5. Implement idempotency using event IDs to avoid duplicate processing.
  6. Log and monitor failures and retries.

FAQ

What is a signed webhook payload?

A signed payload includes a cryptographic signature (usually HMAC) that allows the receiver to verify the request's authenticity and integrity using a shared secret.

How many times should I retry a failed webhook?

Common practice is 3–5 retries with exponential backoff. After that, move the event to a dead-letter queue for manual inspection.

Can I use webhooks without HTTPS?

No, HTTPS is essential to protect the payload and signature from eavesdropping and tampering. Always use TLS.

Ready to test your webhook payloads? Use our JSON formatter to pretty-print and validate JSON payloads quickly.