Client-Side Load Balancer

In this page, we look at the opposite idea: what if the client picks the server itself?

Definition

A client-side load balancer means the client keeps a list of servers and chooses one for each request. There is no separate machine in the middle.

The load balancing logic lives inside the client, usually as a library.

Analogy: instead of asking the host to seat you, you get the full list of tables and walk to a free one yourself.

1Server-side:  Client → Load Balancer → Server 1 / Server 2 / Server 3
2 
3Client-side:  Client (holds the list, picks one) → Server 1 / Server 2 / Server 3

Here, "client" usually means another service, not a browser. For example, ShortLink's RedirectService calling its AnalyticsService.

Let's see how the client knows which servers exist.

How It Works

The client needs two things: a list of servers and a rule to pick one.

The list usually comes from a service registry. This is a shared directory where every server registers its address when it starts, and is removed when it stops.

11. Servers start    → register in the Service Registry
22. Client starts    → asks the registry: "Where is AnalyticsService?"
33. Registry replies → [10.0.0.1, 10.0.0.2, 10.0.0.3]
44. Client picks one → using a rule (for example, round robin)
55. Client → 10.0.0.2   sends the request directly

1              ┌──────────────────┐
2              │ Service Registry │
3              └────────┬─────────┘
4                       │ list of servers
5                       ↓
6RedirectService (client) ──→ AnalyticsService 1 / 2 / 3

The client keeps a local copy of the list and refreshes it every few seconds. So it does not ask the registry for every request.

Here is a simple version in code:

public class ClientSideBalancer {
    private volatile List<String> servers = new ArrayList<>();
    private final AtomicInteger counter = new AtomicInteger(0);

    // Called every few seconds with the latest list from the registry
    public void updateServers(List<String> latestServers) {
        this.servers = new ArrayList<>(latestServers);
    }

    public String pickServer() {
        List<String> current = servers;
        if (current.isEmpty()) {
            throw new IllegalStateException("No servers available");
        }
        int index = Math.floorMod(counter.getAndIncrement(), current.size());
        return current.get(index);
    }
}

Next, what about dead servers?

Health Handling

Here is what breaks. The client holds a list, but a server in that list may have died a few seconds ago. If the client keeps picking it, requests fail.

There are two common fixes:

• Registry removes dead servers. Servers send regular "I'm alive" signals. If the signals stop, the registry removes the server, and clients get the new list on the next refresh.
• Client notices failures. If a call to a server fails or times out, the client skips that server for a while and retries on another one.

1Client → 10.0.0.2   timeout ✗
2Client marks 10.0.0.2 as bad for 30 seconds
3Client → 10.0.0.3   OK ✓

A small gap remains. Until the list refreshes, some requests can hit a dead server. That is why retries matter.

Now let's compare this with a server-side load balancer.

Client-Side vs. Server-Side

Here is the extra-hop gain in numbers:

1Each call through a load balancer adds about 1 ms.
2One user request touches 5 internal services → 5 extra ms.
3 
4Client-side: calls go straight to the server → 0 extra ms.

Advantages

• No extra hop. Calls go directly to the server, so latency is lower.
• No central balancer to fail or fill up. Each client balances its own traffic.
• Cheaper. No load balancer machine to buy and run.
• Smart choices. The client can use its own knowledge, like picking the nearest server.

Limitations

1. More work in every client. Each service needs the balancing code and must keep it updated.

2. Language problem. If services are written in Java, Go, and Python, each needs its own library.

3. Harder to change rules. To change the algorithm, you must update and redeploy many clients. With a server-side balancer, you change one place.

4. Stale lists. A client may use an old list for a few seconds.

5. Needs a registry. The registry is a new part to run, and it must stay available.

6. Not for outside users. You cannot ask every browser or phone app to run your balancing library.

When to choose it:

• Service-to-service calls inside a large system, where many small hops add up.
• When all services use the same language or framework.

When to avoid it:

• Public traffic from browsers and phone apps. Use a server-side load balancer instead.

Summary

1Client-side load balancer → the client holds a server list and picks one itself
2→ The list comes from a service registry and is refreshed often
3→ Dead servers are handled by the registry and by client retries
4→ Advantages: no extra hop, no central balancer, lower cost
5→ Limits: logic in every client, harder updates, needs a registry
6→ Best for: calls between internal services

Both types so far pick a server after the request starts. But what if we could split traffic even earlier, at the moment a user looks up our domain name? That is the DNS-based load balancer, our next topic.