Rate Limiter System Design: Learn Why Rate Limiting Exists (And Why You Need It).

Rate Limiter is NOT Just a Design Pattern — It's a Survival Strategy

In software engineering, we often talk about caching, database indexing, and asynchronous queues to make APIs faster. But rate limiting serves a completely different purpose:

“A Rate Limiter doesn't make APIs faster — it makes systems survive.”

Without a rate limiter, a single rogue script, a DDoS attack, an aggressive web scraper, or a client-side infinite retry loop can consume all database connections and take down your entire infrastructure for every user.

A rate limiter controls the rate at which requests are sent by clients and processed by your server. If incoming requests exceed the configured threshold, excess calls are rejected (typically with an HTTP 429 Too Many Requests status code) or delayed.

1. Token Bucket Algorithm

The Token Bucket is a container with a fixed capacity. Tokens are added into the bucket at a constant refill rate (e.g. 5 tokens per second). When a request arrives, the system checks if a token is available:

• If tokens are available: 1 token is consumed, and the request is processed.
• If bucket is empty: The request is dropped or rejected immediately.
• If bucket is full: Newly generated tokens overflow and are discarded.

2. Leaking Bucket Algorithm

The Leaking Bucket algorithm is similar to the Token Bucket, but it handles traffic through a FIFO (First In, First Out) queue. Imagine a bucket with a small hole at the bottom:

Incoming requests enter the top of the bucket. If the queue has room, the request is queued. If the queue is full, incoming requests overflow and are dropped immediately. Meanwhile, requests leak out the bottom and are processed by the server at a strictly constant, fixed rate.

3. Fixed Window Counter Algorithm

The Fixed Window Counter divides timeline into fixed intervals (e.g., 1-minute blocks like 10:41:00 to 10:42:00). Each window has an atomic counter. Every request increments the counter. If the counter exceeds the allowed threshold (e.g. 10 requests), subsequent requests are rejected. When the clock enters the next minute, the counter resets back to zero.

4. Sliding Window Counter Algorithm

The Sliding Window Counter solves the boundary spike problem of the Fixed Window without the heavy memory penalty of logging every timestamp.

It combines the request count from the current window with a weighted percentage from the previous window:

Estimated Requests = (Current Window Requests) + 
                     (Previous Window Requests * (1 - Current Window Time Elapsed %))

For example, if the previous minute had 10 requests and 30% of the current minute has passed with 3 requests, the estimated rate is: 3 + (10 * 0.70) = 10 requests. If the limit is 10, the next request is rejected.

5. Sliding Window Log Algorithm

The Sliding Window Log algorithm maintains an exact timestamp for every incoming request, usually in a Redis Sorted Set (ZSET).

1. When a request arrives, remove all timestamps older than (Current Time - Window Size).
2. Count how many timestamps remain in the set.
3. If the count is within the limit, add the current timestamp and accept the request.
4. If the count exceeds the limit, reject the request.

-- Redis Lua script for precise Sliding Window Log:
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])
local clearBefore = now - window

-- 1. Remove expired timestamps
redis.call('ZREMRANGEBYSCORE', key, 0, clearBefore)

-- 2. Count current requests
local currentRequests = redis.call('ZCARD', key)

if currentRequests < limit then
    -- 3. Add timestamp and accept
    redis.call('ZADD', key, now, now)
    redis.call('EXPIRE', key, math.ceil(window / 1000))
    return 1
else
    return 0 -- Rejected!
end