Rate Limiting
A rate limiter controls how many requests a client or service can send to a system within a specific period of time.
The basic idea is simple:
A client can make only a certain number of requests within a given time window. Once that limit is reached, additional requests are rejected until the limit becomes available again.
For example:
- A user can create 2 posts per second.
- An IP address can create 10 accounts per day.
- A device can claim a reward 5 times per week.
Why Do We Need Rate Limiting?
Rate limiting helps to:
- Prevent abuse: Stops bots, malicious users, and excessive requests from consuming system resources.
- Prevent server overload: Filters excessive traffic before it reaches backend services.
- Reduce costs: Limits unnecessary requests to servers, databases, and paid third-party APIs.
- Ensure fair usage: Prevents one client from consuming a disproportionate amount of system resources.
Where to Put the Rate Limiter?
A rate limiter can be implemented on the client side or the server side.
1. Client-Side Rate Limiting
Client-side rate limiting is generally not reliable for enforcing limits because the client is controlled by the user.
A malicious user can modify the client or bypass its rate-limiting logic and send requests directly to the server.
Therefore, client-side rate limiting can be useful for preventing unnecessary requests from a legitimate client, but the actual rate limit should be enforced on the server.
2. Server-Side Rate Limiting
A rate limiter can be placed on the server side to control incoming requests before they are processed by the API servers.
The rate limiter checks each request and decides whether it should be allowed or rejected.
For example, assume the API allows 2 requests per second:
429 indicates that the client has sent more requests than the configured limit.
3. Rate Limiter as Middleware
Instead of placing the rate limiter inside the API servers, we can implement it as a middleware layer between the client and the API servers.
This allows the rate limiter to act as a gatekeeper:
- Requests within the limit are forwarded to the API servers.
- Requests exceeding the limit are rejected immediately.
- Excessive traffic does not reach the backend services.
4. Rate Limiter in an API Gateway
In a cloud-based microservices architecture, rate limiting is often implemented at the API Gateway.
An API Gateway acts as a middleware layer between clients and backend services. Apart from rate limiting, it can also handle:
- Authentication
- SSL/TLS termination
- IP whitelisting
- Request routing
- Serving static content
Server-Side vs API Gateway
There is no single correct answer for where to implement rate limiting. The decision depends on the system's technology stack, business requirements, engineering resources, and priorities.
Factors to Consider
-
Technology Stack: Check whether the existing programming language, cache, and infrastructure are suitable for implementing rate limiting on the server side.
-
Rate-Limiting Algorithm: Choose an algorithm that fits the business requirements. A custom server-side implementation gives more control, while a third-party API Gateway may limit the available options.
-
Existing API Gateway: If the system already uses an API Gateway for authentication, IP whitelisting, and request routing, rate limiting can be added there as well.
-
Engineering Resources: Building and maintaining a custom rate-limiting service requires time and engineering effort. If resources are limited, a managed API Gateway can provide rate-limiting capabilities without building the entire system from scratch.
Algorithms for rate limiting
Rate limiting can be implemented using different algorithms, and each of them has distinct pros and cons. Even though this chapter does not focus on algorithms, understanding them at high-level helps to choose the right algorithm or combination of algorithms to fit our use cases. Here is a list of popular algorithms: