On this page

API Gateway

API Gateway is one of the most important components in a microservices architecture. The client should not need to know about every backend service, so we put an API Gateway in front of them as the entry point.

What is an API Gateway?

An API Gateway is a single entry point between the client and the backend microservices. Its main job is:

Accept the client's API request and route it to the correct backend service based on the API endpoint.

For example, GET /api/invoice is routed to the Invoice Microservice.

Why do we need it?

Without a gateway, the client talks directly to every service (Order, Invoice, Sales). This tightly couples the client to the backend architecture.

With a gateway, the client talks only to the gateway, and the gateway handles the routing.

API Gateway vs Load Balancer

Both route traffic, so how are they different? This is a very common interview question. The difference is what they route between.

API GatewayLoad Balancer
Routes requests between different servicesDistributes traffic between instances
Understands API endpointsPrimarily distributes traffic
/order → Order ServiceOrder Service → Order Instance 1/2/3
Can provide authentication, rate limiting, etc.Mainly handles traffic distribution

API Gateway decides WHICH service should handle the request.

Load Balancer decides WHICH INSTANCE of that service should handle the request.

API Composition

A single page may need data from multiple services. For example, a My Orders page may need data from the Product, Invoice, Review, and Recommendation services. If the client calls each one, it has to coordinate all the calls and combine the responses itself.

API Composition solves this. The gateway (or a dedicated service) calls the required services, combines their responses, and returns one response to the client.

For example, the client calls:

1GET /api/my-orders

The gateway gathers the data from the services and returns:

json
1{
2 "product": {},
3 "invoice": {},
4 "reviews": [],
5 "recommendations": []
6}

The client makes only one API call.

API Composition = Multiple backend API calls → One client response

Authentication

The gateway already sits between the client and all the microservices, so we can authenticate at the gateway instead of in every microservice.

For example, using an OAuth 2.0-style flow:

Without this, each microservice repeats the same authentication logic. With the gateway, the request is verified at one place, and invalid or unauthorized requests are rejected before they reach any service.

Authenticate at the entry point, then allow valid requests to continue.

Rate Limiting

Authentication controls who can access the API. But what if a valid client sends too many requests? For example, 10,000 requests at once can overload the backend.

So the gateway also provides rate limiting and throttling, using these mechanisms:

  • Burst limits
  • API throttling
  • IP-based blocking
  • API queues

Burst Limit

A burst limit controls how much traffic can be handled during a sudden spike.

1Burst Limit = 500

If incoming traffic goes above the limit, the gateway rejects the extra requests before they reach the backend, with:

1HTTP 429
2Too Many Requests
1 Request Request Request
2 | | |
3 +----------+----------+
4 |
5 API Gateway
6 |
7 Burst Limit
8 |
9 Limit exceeded
10 |
11 ▼
12 HTTP 429

API Throttling

Burst limits handle sudden spikes. Throttling answers a more specific question:

How many requests is a particular user or application allowed to make?

1/api/invoice
2
3Maximum:
410 requests / minute / user
1Request 1 ✓
2Request 2 ✓
3...
4Request 10 ✓
5Request 11 ✗

The 11th request is blocked because the user went over the allowed rate. Throttling can be applied per API, user, application, or request rate.

IP-Based Blocking

The gateway can block requests from a specific IP address. This is another layer of traffic protection.

1Client (IP: X.X.X.X)
2 |
3 ▼
4API Gateway
5 |
6 | IP blocked
7 ▼
8Request rejected

API Queues

When a huge number of requests arrive together, we do not let them all hit the backend at once. The rate limiter controls how many requests enter the system. The accepted requests wait in a queue, and the queue releases them gradually based on the capacity of the backend service.

For example, in an e-commerce flash sale, thousands of users try to place an order at the same time. The queue feeds the Order Service at the speed it can handle, so the spike does not overwhelm it.

Service Discovery

The gateway now knows which microservice should get the request. But where is that microservice running?

Services scale up and down, so instances are created and removed, and their IP addresses and ports change. Service Discovery (also called the Service Registry) keeps track of the current location of every service instance.

How it works

  1. The client sends /api/order.
  2. The gateway decides this belongs to the Order Service.
  3. The gateway asks the registry for the healthy instances of the Order Service.
  4. The gateway forwards the request to one of them, usually through a Load Balancer.

How the registry stays up to date

Approach 1: Self-registration. Whenever a microservice starts, it registers itself with Service Discovery.

Approach 2: Health checks. Service Discovery keeps checking the services. If a service stops responding, that instance is removed from the active list, so only healthy locations remain.

Common technologies: Eureka, Consul, ZooKeeper, etcd, and Kubernetes Service Discovery. In Spring Cloud, Eureka acts as the service registry.

Other API Gateway Responsibilities

Besides routing, composition, authentication, rate limiting, and service discovery, a gateway can also handle:

  • Request and response transformation
  • Caching
  • Centralized logging

If the Gateway Is a Single Entry Point, How Does It Handle Millions of Requests?

Won't the API Gateway itself become a bottleneck or a single point of failure?

No. "Single entry point" is a logical idea, not a single server. In production, the gateway runs on many instances, across many Availability Zones, and often across many regions.

Multiple Gateway Instances

Many gateway instances run in parallel behind a load balancer. The load balancer spreads requests across them and stops sending traffic to unhealthy ones. When traffic grows, auto-scaling adds more instances.

Multiple Availability Zones

A region contains several isolated Availability Zones (AZs). Each AZ runs its own gateway and service instances, and traffic is spread across the AZs.

If AZ1 fails, traffic is redirected to AZ2, and the application keeps serving requests. This gives high availability and fault tolerance inside a region.

Multiple Regions

Multi-AZ protects against failures inside a region. If the entire region goes down, we need another level of redundancy.

The application is deployed in several regions. Each region has its own gateway, load balancers, service discovery, and microservice instances. If one region fails, traffic is redirected to another healthy region.

DNS-Based Traffic Distribution

With many regions, who decides which region gets the client's request? DNS and traffic-routing services such as AWS Route 53 and Azure Traffic Manager do. They choose a region using rules based on latency, geographic location, or compliance requirements. The request then goes to the gateway in that region.

Complete API Gateway Architecture

ComponentMain Responsibility
DNS / Traffic ManagerChoose the appropriate region
API GatewayRoute request to the correct service
AuthenticationVerify the client/token
Rate LimitingControl request rate
Service DiscoveryFind current service locations
Load BalancerDistribute traffic across service instances
MicroserviceProcess the actual business request

One-Line Mental Model

DNS chooses the region → API Gateway chooses the service → Service Discovery finds its location → Load Balancer chooses the instance → Microservice processes the request.

Next TopicAuthentication