Webhooks Explained: Signed Payloads and Retries

You've just integrated a third-party service that sends webhooks to your endpoint. Everything works in testing, but in production you notice missed events, duplicate deliveries, and occasional security concerns. Webhooks are simple in concept but tricky in practice—especially when it comes to verifying authenticity and handling failures.

This guide explains how to secure webhooks with signed payloads and implement robust retry mechanisms, so you can build reliable integrations.

What Are Webhooks?

Webhooks are user-defined HTTP callbacks. When an event occurs in a source system (e.g., a payment is completed), it sends an HTTP POST request to a URL you specify, containing data about the event. This allows real-time communication between services without polling.

But webhooks introduce challenges: how do you know the request genuinely came from the expected sender? What if your endpoint is temporarily down? That's where signed payloads and retries come in.

Securing Webhooks with Signed Payloads

A signed payload uses a cryptographic signature to prove the request originated from the sender and wasn't tampered with. The common approach is HMAC (Hash-based Message Authentication Code) with a shared secret.

How HMAC Signatures Work

The sender computes an HMAC over the request body (and sometimes headers like timestamp) using a secret key. The signature is sent in a header (e.g., X-Signature). Your server recomputes the HMAC and compares it to the received signature. If they match, the request is authentic.

Here's a minimal example in Python using Flask:

import hmac
import hashlib
from flask import Flask, request, abort

app = Flask(__name__)
SECRET = b'your_shared_secret'

@app.route('/webhook', methods=['POST'])
def webhook():
    signature = request.headers.get('X-Signature')
    if not signature:
        abort(400, 'Missing signature')
    body = request.get_data()
    expected = hmac.new(SECRET, body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, signature):
        abort(401, 'Invalid signature')
    # Process the event
    return '', 200

Key points:

• Always use a constant-time comparison function (like hmac.compare_digest) to prevent timing attacks.
• Include a timestamp in the signed data to prevent replay attacks; reject requests older than a few minutes.
• Keep the secret secure—never expose it client-side.

Additional Security Measures

• HTTPS: Always use HTTPS to protect the payload in transit.
• IP allowlisting: If the provider publishes IP ranges, restrict requests to those IPs.
• Idempotency: Use event IDs to deduplicate, as retries may cause duplicates.

Implementing Reliable Retries

Webhook consumers can fail: your server might be down, or processing might time out. Senders should retry failed deliveries with exponential backoff. As a consumer, you must handle retries gracefully.

On the Sender Side (If You're Building a Webhook System)

1. Return a 2xx quickly: Acknowledge receipt before processing to avoid timeouts. Use a queue for async work.
2. Retry on failure: If no 2xx is received within a timeout (e.g., 5 seconds), retry with increasing delays (e.g., 1s, 5s, 30s, 5m, 30m).
3. Limit retries: After a maximum number of attempts (e.g., 5), mark the delivery as failed and alert.
4. Include a unique event ID: So consumers can deduplicate.

On the Consumer Side (Your Endpoint)

• Respond fast: Return 2xx immediately and process asynchronously if needed.
• Be idempotent: Use the event ID to avoid processing the same event twice.
• Handle duplicates: Store processed event IDs for a reasonable period.
• Log failures: If processing fails, return 5xx to trigger a retry, but ensure you don't get stuck in a loop.

Comparison: Signed Payloads vs. Basic Auth

Testing Webhooks Locally

During development, you need a public URL to receive webhooks. Tools like ngrok or localtunnel can expose your local server. Always test signature verification and retry behavior with simulated failures.

When inspecting webhook payloads, you might need to format JSON for readability. The JSON Formatter can help you pretty-print and validate the JSON body.

FAQ

What is a signed webhook payload?

A signed payload includes a cryptographic signature (often HMAC) that allows the receiver to verify the request's authenticity and integrity.

How many times should a webhook be retried?

There's no fixed number; common practices range from 3 to 10 retries with exponential backoff. The key is to balance reliability with resource usage and alert on persistent failures.

Can I use webhooks without signatures?

It's not recommended. Without signatures, you can't verify the sender, making your endpoint vulnerable to spoofing. Always use signatures in production.

Ready to debug your webhook payloads? Try our JSON Formatter to quickly validate and format JSON data.