Scaling Backend Systems
The Part's closing chapter — what happens when a system that worked fine for a thousand users meets a hundred thousand.
5.15.1Definition
Scaling is the set of techniques that let a backend handle increasing load — more users, more requests, more data — without a proportional degradation in performance or reliability. It splits broadly into vertical scaling (a bigger server) and horizontal scaling (more servers), each with different implications for the architecture choices covered throughout this Part.
5.15.2Why It Exists
Every system has a ceiling — a point where a single database, a single server, or a single process can no longer keep up with demand. Scaling practices exist because that ceiling is reached far more often than beginners expect, and the architectural decisions needed to raise it (statelessness, caching, sharding) are far cheaper to build in from early on than to retrofit under production load.
5.15.3Vertical vs. Horizontal Scaling
| Vertical scaling | Horizontal scaling | |
|---|---|---|
| Approach | Bigger single machine (more CPU/RAM) | More machines behind a load balancer |
| Ceiling | Hardware limits eventually cap it | Effectively unlimited, given the right architecture |
| Requirement | None — works with any architecture | Requires stateless application servers (5.3), shared session/cache storage |
| Cost curve | Rises steeply at the high end | Scales roughly linearly with added capacity |
5.15.4Common Mistakes
- Storing session state in server memory (5.3), preventing horizontal scaling entirely because a user's next request may land on a different server that knows nothing about their session.
- Scaling the application layer while ignoring the database, which remains a single bottleneck no number of additional web servers can relieve.
- Premature horizontal scaling — introducing distributed-systems complexity (1.5) for load a single well-optimized server could handle for years.
- No caching layer (5.9) before scaling out, multiplying database load across more servers instead of reducing it in the first place.
5.15.5Best Practices
- Keep application servers stateless from day one — even before scaling is a concern — so horizontal scaling remains an option later without a rewrite.
- Exhaust caching (5.9), query optimization, and vertical scaling before reaching for horizontal complexity.
- Scale the database deliberately — read replicas, connection pooling, sharding — as its own distinct problem from application server scaling.