Horizontal Scaling
Hop's dashboard server is at 100% CPU. There are two ways to get more capacity: buy a bigger machine, or buy more machines. The second is how large systems grow — but it only works if Hop's servers are built for it.
The rule
Grow by adding machines, and keep no state on any one of them.
Vertical scaling (scaling up) means a bigger machine: more CPU, more memory.
- Simple — no code changes.
- But there is a biggest machine you can buy, prices rise steeply near the top, and it is still one box that can fail.
Horizontal scaling (scaling out) means more machines of the same kind behind a load balancer.
- Capacity grows by adding servers; you can also remove them when traffic drops.
- Losing one server is survivable — the others carry on.
- But it only works if any server can handle any request.
That last condition is the heart of it. A server is stateless when it keeps nothing about a user between requests in its own memory or disk. Anything that must last — sessions, uploaded files, data — goes into shared places every server can reach: a database, a cache service, object storage, or a signed token the client sends with each request.
The shortcut
Before adding servers, ask: "If this user's next request lands on a different server, does anything break?" If yes, find what that server is keeping to itself and move it to a shared store.
Try adding servers
Five dashboard servers behind round robin. Users start getting logged out at random.
Sticky sessions (a hash that pins each user to one server) can hide the problem, but when that server dies its users are logged out, and load spreads unevenly. Moving state out is the real fix.
Remember too that stateless servers push the pressure onto the shared stores. Scaling out Hop's servers is easy; the database behind them is where the next lessons of system design begin.
Drills
Answer fast — trust the rule.
Which is horizontal scaling?
What is a main weakness of vertical scaling?
After scaling out, users are randomly logged out. What is the likely cause?
Where should uploaded files go so that any server can serve them?
Why are sticky sessions only a partial fix?
Remember
- Grow by adding machines, and keep no state on any one of them.
- Shortcut: "if the next request lands elsewhere, does anything break?" Move that state to a shared store.
- Trap: sticky sessions hide the problem instead of fixing it.