Server-Side Load Balancer

In the previous page, we grouped load balancers by where the balancing happens. In this page, we look at the most common type: the server-side load balancer. It is what ShortLink uses today.

Definition

A server-side load balancer is a separate machine or service that sits in front of your servers. Clients send every request to it, and it picks one server to handle that request.

The client does not choose. The client does not even know how many servers exist. It only knows one address.

Analogy: the host at a restaurant. Customers talk to the host, and the host sends each one to a free table. Customers never walk into the kitchen to pick a cook.

1                          ┌→ App Server 1
2Clients → Load Balancer → ├→ App Server 2
3                          └→ App Server 3

Let's follow one request to see how it works.

How It Works

A user opens sl.io/abc123. The load balancer sits between the user and the servers.

11. Client → Load Balancer         "GET sl.io/abc123"
22. Load Balancer checks its list  (which servers are healthy?)
33. Load Balancer picks one        (using a rule, like round robin)
44. Load Balancer → App Server 2   forwards the request
55. App Server 2 → Load Balancer   "redirect to the long link"
66. Load Balancer → Client         sends the answer back

The key point: the load balancer is in the middle of every request and every response. That is why it can see what is happening and react quickly, for example when a server dies.

Next, let's see what jobs it does besides picking a server.

What the Load Balancer Does

A server-side load balancer usually does four jobs:

• Spreads requests. It uses a rule such as round robin or least connections so no server is overloaded.
• Checks health. It keeps asking each server "are you alive?" and skips the ones that do not answer.
• Hides the servers. Users only see one address. We can add, remove, or replace servers without users noticing.
• Gives one place for control. Settings like timeouts and limits live in one machine, not in every client.

Here is the math for ShortLink:

1Peak traffic:      3,000 RPS
2Servers:           5
3Each server gets:  3,000 / 5 = 600 RPS
4One server limit:  1,000 RPS → 60% busy ✓

Now let's see what this looks like in a real setup.

Example Setup

Most teams do not write a load balancer from scratch. They use a tool like NGINX or HAProxy, and write a short config file. Here is a simple NGINX example for ShortLink:

1upstream shortlink_servers {
2    least_conn;                  # rule: pick the server with fewest connections
3    server 10.0.0.1:8080;
4    server 10.0.0.2:8080;
5    server 10.0.0.3:8080;
6}
7 
8server {
9    listen 80;
10 
11    location / {
12        proxy_pass http://shortlink_servers;   # forward to the group
13    }
14}

In plain words: "Make a group of three servers. Use least connections. Send every request on port 80 to that group."

To add a fourth server, we add one line and reload. Users notice nothing. Cloud services like AWS Elastic Load Balancing do the same job, but you set it up in a console instead of a file.

So what do we gain from this design?

Advantages

But it is not free.

Limitations

1. Extra hop. Every request passes through one more machine. This adds a small delay.

2. Single point of failure. If the load balancer dies and there is only one, ShortLink is down. We fix this by running two (active-passive or active-active), as we saw before.

3. Possible bottleneck. All traffic goes through it. If traffic grows beyond its capacity, everything slows down, even if the servers are fine.

1Say the load balancer handles 50,000 RPS at most.
2ShortLink grows to 60,000 RPS.
3Servers are fine, but the load balancer is full → users see slow responses ✗

4. Extra cost. You pay for the load balancer and its backup, and someone must run and watch it.

When to choose it:

• Most web apps and APIs, where users or browsers call your system from outside.
• When you want clients to stay simple.
• When you want one place to manage traffic rules.

When to look at other types:

• Service-to-service calls inside a big system, where an extra hop for every call adds up. Here a client-side load balancer can work better.

Summary

1Server-side load balancer → a separate machine in front of the servers
2→ Clients send every request to one address
3→ It picks a healthy server using a rule and forwards the request
4→ It spreads load, checks health, and hides the servers
5→ Advantages: simple clients, central control, easy to add servers
6→ Limits: extra hop, single point of failure, possible bottleneck
7→ Best for: most web apps and APIs used from outside

In this type, a machine in the middle picks the server. But what if the client picked the server itself and we skipped the middle machine? That is the client-side load balancer, our next topic.