HLD Fundamentals #4: How Systems Scale: From 0 to 100 Million Users

One of the most common system design interview questions is: "How would you scale a web application from 100 users to 100 million users?" The answer is rarely a single technology. Instead, systems evolve through multiple stages, with each stage solving a specific bottleneck. This article walks through the typical evolution of a scalable system and explains why , how , and when each component is introduced. 1. Single Server Why Start Here? Every application starts simple. In the beginning: Traffic is low Development speed matters more than scalability Infrastructure costs should be minimal What Is It? A single machine handles everything: Frontend Backend Database Users | v Single Server ├── Application └── Database How Does It Work? User sends request. Application processes request. Database stores and retrieves data. Response is returned. Everything happens on one machine. Problem As traffic grows: CPU becomes overloaded Memory becomes insufficient Database competes with application for resources A single server becomes a bottleneck. Interview One-Liner A single server architecture is simple and cost-effective but becomes a bottleneck as traffic and resource usage increase. 2. Application and Database Separation Why Do We Need It? The application and database have different workloads. Application Server: Uses CPU Handles business logic Database Server: Uses memory and storage Handles queries Keeping them together causes resource contention. How Does It Work? Move the database to a separate machine. Users | v Application Server | v Database Server Benefits Independent scaling Better resource utilization Improved performance Example Suppose an e-commerce website receives thousands of requests. The application handles: Authentication Order processing API responses The database handles: Product data Orders User information Separating them prevents one workload from affecting the other. Interview One-Liner Separating the application and database allows each layer to scal