Webhooks Explained: Signed Payloads and Retries

You've just integrated a payment gateway or a CI/CD service, and now you need to receive real-time events. Webhooks are the standard solution, but they come with two critical challenges: verifying that the request genuinely comes from the sender and ensuring that events are not lost if your server is temporarily unavailable. This article explains how to implement signed payloads and retry mechanisms to build a robust webhook receiver.

What Are Webhooks?

A webhook is an HTTP callback: instead of your application polling an API for updates, the service sends an HTTP POST request to a URL you provide whenever an event occurs. This push model reduces latency and server load. However, because the request originates from an external service, you must treat it as untrusted input.

Why Sign Payloads?

Without verification, anyone who discovers your webhook endpoint could send fake events, potentially triggering unauthorized actions. Signing payloads with a shared secret allows you to confirm the sender's identity and ensure the payload hasn't been tampered with. The most common method is HMAC (Hash-based Message Authentication Code) using SHA-256.

How HMAC Signing Works

The sender computes an HMAC digest of the raw request body using a secret key and includes it in a header (e.g., X-Signature). Your server recomputes the HMAC with the same secret and compares it to the received signature. If they match, the request is authentic.

// Node.js example: verifying a webhook signature
const crypto = require('crypto');

function verifySignature(payload, signature, secret) {
  const hmac = crypto.createHmac('sha256', secret);
  hmac.update(payload);
  const digest = hmac.digest('hex');
  // Use timingSafeEqual to prevent timing attacks
  return crypto.timingSafeEqual(
    Buffer.from(digest, 'hex'),
    Buffer.from(signature, 'hex')
  );
}

Important: Always use a constant-time comparison function (like timingSafeEqual) to avoid leaking information through timing differences. Also, compute the HMAC over the raw body, not the parsed JSON, because re-serializing can change the byte sequence.

Implementing Retries

Even with signatures, network issues or server errors can cause webhook delivery to fail. A reliable sender will retry failed deliveries, typically with exponential backoff. As a receiver, you should design your endpoint to be idempotent and return appropriate status codes.

Retry Strategy from the Sender's Perspective

• Retry on 5xx and network errors: These indicate temporary problems.
• Do not retry on 4xx: Client errors (e.g., 400 Bad Request) are usually permanent; retrying won't help.
• Use exponential backoff: Wait longer between each retry to avoid overwhelming the receiver.
• Set a maximum number of retries: After a limit, mark the delivery as failed and alert.

Designing an Idempotent Receiver

Because retries can cause duplicate deliveries, your receiver must handle the same event multiple times without adverse effects. Include a unique event ID in the payload (or header) and store processed IDs. If an event ID has already been processed, return a 200 OK immediately without reprocessing.

// Pseudocode for idempotent webhook handling
function handleWebhook(event) {
  if (processedEvents.has(event.id)) {
    return { status: 200, body: 'Already processed' };
  }
  // Process event...
  processedEvents.add(event.id);
  return { status: 200, body: 'Success' };
}

Best Practices for Webhook Security and Reliability

• Use HTTPS: Always serve your webhook endpoint over TLS to prevent eavesdropping.
• Validate the timestamp: If the signature includes a timestamp, reject requests older than a few minutes to prevent replay attacks.
• Respond quickly: Acknowledge receipt with a 2xx status before doing heavy processing. Queue the work asynchronously.
• Log everything: Keep logs of received webhooks, including headers and payloads (redact sensitive data), to debug issues.
• Rotate secrets periodically: Support multiple active secrets to allow seamless rotation.

Comparison: Polling vs. Webhooks

FAQ

How do I verify a webhook signature?

Compute an HMAC of the raw request body using the shared secret, then compare it to the signature in the header using a constant-time comparison function.

What should my webhook endpoint return?

Return a 2xx status code (e.g., 200 OK) as soon as you've received the event. If you return 4xx or 5xx, the sender may retry or mark the delivery as failed.

Can I use webhooks without signatures?

It's not recommended. Without signatures, you cannot verify the sender, leaving your endpoint vulnerable to fake requests. Always use signatures in production.

When debugging webhook payloads, JSON formatting is essential. Use our JSON formatter to pretty-print and validate incoming JSON data quickly.