Webhooks Explained: Signed Payloads and Retry Strategies

Backend2026-11-12TryQuickToolBox

You've built an API that sends webhooks to notify external services about events. But what happens when the receiving endpoint is down? Or when a malicious actor tries to forge a webhook request? Without proper safeguards, your webhooks can become a source of data inconsistency and security vulnerabilities. This article explains how to implement signed payloads and retry mechanisms to make your webhooks reliable and secure.

What Are Webhooks and Why Do They Need Security?

Webhooks are user-defined HTTP callbacks that are triggered by specific events. They enable real-time communication between systems, such as notifying a payment service when an order is placed. However, because webhooks are publicly accessible endpoints, they are susceptible to spoofing and replay attacks. Additionally, network issues or receiver downtime can cause missed events. Therefore, two critical aspects must be addressed: authenticity (via signed payloads) and reliability (via retries).

Signed Payloads: Ensuring Authenticity and Integrity

A signed payload allows the receiver to verify that the webhook originated from a trusted source and that the data hasn't been tampered with. This is typically achieved using HMAC (Hash-based Message Authentication Code) with a shared secret.

How HMAC Signing Works

The sender and receiver share a secret key. For each webhook request, the sender computes an HMAC signature using the secret and the request payload (often including a timestamp to prevent replay attacks). The signature is sent in a header, such as X-Signature. The receiver recomputes the signature and compares it with the received one. If they match, the request is authentic.

Here's an example in Python using HMAC-SHA256:

import hmac
import hashlib
import time

secret = b'your_shared_secret'
payload = b'{"event":"order.created","id":123}'
timestamp = str(int(time.time())).encode()
message = timestamp + b'.' + payload
signature = hmac.new(secret, message, hashlib.sha256).hexdigest()

# Send headers: X-Signature: signature, X-Timestamp: timestamp

On the receiver side, verify the signature and check the timestamp to ensure the request is recent (e.g., within 5 minutes). This prevents replay attacks.

Best Practices for Signed Payloads

Retry Mechanisms: Ensuring Reliable Delivery

Even with signed payloads, network failures or receiver errors can cause webhook delivery to fail. A robust retry strategy ensures that events are eventually delivered without overwhelming the receiver.

Designing a Retry Policy

A common approach is exponential backoff with jitter. For example, retry after 1 minute, then 5 minutes, then 30 minutes, up to a maximum number of attempts (e.g., 5). Include a unique event ID in the payload so the receiver can deduplicate.

Example retry schedule:

AttemptDelay
1Immediate
21 minute
35 minutes
430 minutes
52 hours

After exhausting retries, move the event to a dead-letter queue for manual inspection.

Idempotency: Handling Duplicate Deliveries

Because retries can cause duplicate deliveries, receivers must be idempotent. Include an event ID in the payload and have the receiver store processed IDs to ignore duplicates. Alternatively, design operations to be naturally idempotent (e.g., using PUT instead of POST).

Implementing Webhooks: A Step-by-Step Guide

  1. Define events and payload structure: Document the events your system emits and the JSON schema for each.
  2. Generate a secret: Create a unique secret per webhook endpoint and share it securely.
  3. Sign each request: Compute HMAC over the payload and timestamp, include in headers.
  4. Implement retries with backoff: Use a queue system to schedule retries.
  5. Provide a way to verify signatures: Document the verification process for receivers.
  6. Monitor and log: Track delivery attempts and failures for debugging.

Common Pitfalls and How to Avoid Them

FAQ

What is a signed webhook payload?

A signed webhook payload includes a cryptographic signature (e.g., HMAC) that allows the receiver to verify the sender's authenticity and data integrity.

How many times should I retry a failed webhook?

Typically 3 to 5 retries with exponential backoff. The exact number depends on your SLA and the criticality of the event.

Can I use JWT instead of HMAC for webhook signatures?

Yes, JWT can be used, but HMAC is simpler and more common for webhook signing. JWT adds overhead and is better suited for authentication tokens.

Ready to test your webhook payloads? Use our JSON formatter to validate and beautify your JSON payloads before sending.