Redis Use Cases: Caching, Queues and Rate Limiting

You added Redis to your stack, but you're only using it as a simple key-value store. Redis offers powerful data structures and commands that can solve caching, queuing, and rate limiting elegantly. This guide shows you how to implement these patterns effectively.

Why Redis for These Use Cases?

Redis is an in-memory data store known for its speed and versatility. It supports strings, hashes, lists, sets, sorted sets, and more. Its atomic operations and built-in expiration make it ideal for caching, queues, and rate limiting. Plus, it's easy to integrate with any language.

Caching with Redis

Caching is the most common Redis use case. You store frequently accessed data in Redis to reduce database load and speed up responses.

Basic Cache-Aside Pattern

In the cache-aside pattern, your application checks Redis first. If the data isn't there (cache miss), it fetches from the database, then stores it in Redis for next time.

import redis
import json

r = redis.Redis(host='localhost', port=6379, db=0)

def get_user(user_id):
    cache_key = f'user:{user_id}'
    cached = r.get(cache_key)
    if cached:
        return json.loads(cached)
    user = fetch_user_from_db(user_id)  # your DB call
    r.setex(cache_key, 3600, json.dumps(user))  # cache for 1 hour
    return user

Best practices:

• Always set an expiration time (TTL) to avoid stale data.
• Use meaningful key names, like user:123:profile.
• Handle cache invalidation: when data changes, update or delete the cache entry.
• Consider using Redis hashes for storing multiple fields of an object to save memory.

Advanced Caching Patterns

For high-traffic applications, consider these patterns:

• Write-through: Update cache and database together.
• Write-behind: Update cache first, then asynchronously update the database.
• Refresh-ahead: Proactively refresh cache entries before they expire.

Queues with Redis

Redis lists and streams are perfect for building simple message queues. Use lists for basic job queues and streams for more complex event processing.

Simple Job Queue with Lists

Use LPUSH to add jobs and BRPOP to process them. BRPOP blocks until a job is available, making it efficient for workers.

# Producer
r.lpush('jobs', json.dumps({'task': 'send_email', 'to': 'user@example.com'}))

# Consumer (worker)
while True:
    _, job_data = r.brpop('jobs', timeout=30)
    job = json.loads(job_data)
    process_job(job)

Best practices:

• Use separate lists for different job types or priorities.
• Implement a dead-letter queue for failed jobs.
• Use BRPOPLPUSH (or BLMOVE in Redis 6.2+) to atomically move jobs to a processing list, ensuring reliability.
• Monitor queue length with LLEN to scale workers.

Redis Streams for Event Sourcing

Redis Streams (introduced in Redis 5.0) provide a more robust queue with consumer groups, acknowledgments, and replay. Use XADD to add events and XREADGROUP to consume them.

# Add event
r.xadd('events', {'type': 'order_placed', 'order_id': '123'})

# Consumer group
r.xgroup_create('events', 'order_processors', id='0', mkstream=True)
while True:
    events = r.xreadgroup('order_processors', 'worker1', {'events': '>'}, count=10, block=5000)
    for event in events:
        process_event(event)
        r.xack('events', 'order_processors', event.id)

Rate Limiting with Redis

Rate limiting protects your APIs from abuse and ensures fair usage. Redis's atomic operations make it ideal for counting requests.

Fixed Window Rate Limiting

Increment a counter for each user and set an expiration. If the count exceeds the limit, block the request.

def is_allowed(user_id, limit=100, window=60):
    key = f'rate_limit:{user_id}'
    current = r.incr(key)
    if current == 1:
        r.expire(key, window)
    return current