Load Balancing Algorithms

Hop's load balancer receives a request and has three servers to choose from. Which one? The rule it uses is the load balancing algorithm, and the right rule depends on how similar your servers and your requests are.

The rule

Equal work, take turns; uneven work, pick the least busy.

The common algorithms:

• Round robin — servers take turns: 1, 2, 3, 1, 2, 3… No bookkeeping. Ideal when servers are identical and requests are short and similar.
• Weighted round robin — turns in proportion to a weight. Weights 2, 1, 1 send half the traffic to the first server. For pools with machines of different power.
• Least connections — send to the server with the fewest requests in progress right now. Ideal when some requests take far longer than others, because it reacts to real load.
• Least response time — a variation that also favours the server answering fastest.
• Hash-based — hash something about the request (the client's IP, a user id, the short code) to choose the server, so the same key always reaches the same server. Useful when a server keeps a warm local cache for certain keys, or when a client must stay on one server ("sticky sessions").
• Random — surprisingly decent at scale; picking the less busy of two random servers works even better.

The shortcut

Ask two questions in order:

1. "Must the same key reach the same server?" Yes → hash-based.
2. Otherwise, "Are requests similar in length?" Yes → round robin (weighted if servers differ). No → least connections.

The trap is round robin on uneven work. Three long imports can land on one server by bad luck of the rotation while the others sit idle, because round robin counts turns, not load.

Drills

Answer fast — trust the rule.

Remember

• Equal work, take turns; uneven work, pick the least busy.
• Shortcut: sticky key → hash; similar requests → round robin (weighted if servers differ); uneven → least connections.
• Trap: round robin counts turns, not load.