Webhooks Explained: Signed Payloads and Retries

Why Webhooks Need Signatures and Retries

You've built an endpoint to receive webhooks from a payment provider or SaaS platform. Everything works in testing, but in production you face two problems: how do you know the request really came from the provider, and what happens when your server is down or slow? Without signed payloads, anyone can forge a request. Without retries, a single network blip can lose an event forever.

This guide explains how to implement both correctly. We'll cover HMAC signing, verification steps, retry strategies, and idempotency—the three pillars of reliable webhook handling.

How Webhook Signatures Work

A webhook signature is a cryptographic hash that proves the payload was created by someone who knows a shared secret. Most providers use HMAC-SHA256. The sender computes the signature over the raw request body (sometimes with a timestamp) and includes it in a header. The receiver recomputes it and compares.

Here's a typical flow:

1. Provider and consumer share a secret key (configured in the provider's dashboard).
2. Provider creates the payload, computes HMAC-SHA256(secret, payload), and sends it in a header like X-Signature.
3. Consumer receives the request, reads the raw body, recomputes the HMAC, and compares using a constant-time function.
4. If they match, process the event; otherwise, reject with 401.

Important: Always use the raw request body as received, not a parsed-and-restringified version. JSON serialization can change whitespace and key order, breaking the signature.

Adding Timestamps to Prevent Replay Attacks

A signed payload alone can be replayed by an attacker who captures a valid request. To mitigate, include a timestamp in the signed data and reject requests older than a few minutes. For example:

signed_payload = timestamp + "." + raw_body
signature = HMAC-SHA256(secret, signed_payload)

The consumer checks that the timestamp is within an acceptable window (e.g., 5 minutes) and that the signature matches.

Implementing Signature Verification

Let's look at a concrete example in Python using Flask. We'll verify a signature sent in the X-Signature header.

import hmac
import hashlib
from flask import Flask, request, abort

app = Flask(__name__)
SECRET = b'your_shared_secret'

@app.route('/webhook', methods=['POST'])
def webhook():
    signature = request.headers.get('X-Signature')
    if not signature:
        abort(401)

    raw_body = request.get_data()
    expected = hmac.new(SECRET, raw_body, hashlib.sha256).hexdigest()

    if not hmac.compare_digest(expected, signature):
        abort(401)

    # Process the event
    event = request.json
    # ... handle event ...
    return '', 200

Key points:

• Use hmac.compare_digest (or equivalent) for constant-time comparison to avoid timing attacks.
• Access the raw body via request.get_data() before any parsing.
• Return 200 quickly; do heavy processing asynchronously.

Designing a Retry Strategy

Even with signatures, webhooks can fail: your server might be down, return a 500, or time out. Providers typically retry with exponential backoff. As a consumer, you should also handle retries gracefully.

Here's what to consider:

• Idempotency: Each event should have a unique ID. Store processed IDs to avoid duplicate processing when retries occur.
• Respond fast: Acknowledge receipt with a 2xx status within a few seconds. Queue the event for background processing.
• Retry logic on your side: If you call downstream services, implement retries with backoff and circuit breakers.

Most providers retry on non-2xx responses or timeouts, often for up to 24 hours with increasing intervals. Check your provider's documentation for specifics.

Example: Idempotent Event Handler

Use a database table to track processed event IDs. Here's a simplified pattern:

def handle_event(event_id, payload):
    if db.exists('processed_events', event_id):
        return  # already handled
    # process payload
    db.insert('processed_events', event_id)
    # ... actual work ...

For high throughput, use a unique constraint on the event ID and catch duplicate key errors.

Comparison: Signed vs Unsigned Webhooks

Common Pitfalls and Best Practices

• Don't parse before verifying: Always verify the signature on the raw body first.
• Use HTTPS: Signatures protect integrity, but HTTPS protects confidentiality.
• Rotate secrets: Support multiple active secrets during rotation to avoid downtime.
• Log failures: Record signature mismatches and retries for debugging.
• Test with real payloads: Use tools like webhook.site or your provider's CLI to simulate events.

FAQ

What is a signed webhook payload?

A signed payload includes a cryptographic signature (usually HMAC) that lets the receiver verify the request came from a trusted source and wasn't tampered with.

How do I handle webhook retries without duplicating actions?

Make your handler idempotent by tracking event IDs. Before processing, check if the ID was already handled; if so, return success without repeating the work.

Can I use webhooks without signatures?

It's not recommended for production. Without signatures, you can't distinguish legitimate requests from forged ones. Always verify signatures when available.

When debugging webhook payloads or analyzing server logs, the JSON Formatter can help you inspect and validate JSON structures quickly.