System Design Fundamentals¶

Start here if you can ship an API + database and need a method for everything larger.

Why This Exists¶

Most engineers fail system design interviews for one reason: they start drawing boxes before they know what they are building.

The prompt is "design Twitter." The candidate immediately draws a load balancer, three app servers, a database, and Redis. Forty minutes later the interviewer asks "what happens when Justin Bieber tweets?" and the whole design collapses — because it was never derived from the workload. It was recalled from a diagram someone saw once.

The fix is a method. Given any unfamiliar system, you should be able to ask the right questions, put numbers on the answers, and let those numbers force the architecture. Boxes come last, not first.

Mental Model¶

Every design decision in this academy flows one direction:

Requirements ──→ Numbers ──→ Bottleneck ──→ Architecture
     │              │             │              │
  "what and     "how much     "what breaks   "the smallest
   for whom"     per second"    first"        thing that
                                              survives it"

Reverse any arrow and you get cargo-cult design. "We use Kafka" is an answer looking for a question. The bottleneck is what earns you the box.

The Three Questions That Drive Everything¶

Almost every architecture in this curriculum falls out of three numbers.

1. How many requests per second? Tells you whether a naive single-node or single-Postgres default is still safe to assume — not which architecture to draw. Benchmark representative queries before naming a mechanism.

2. What is the read:write ratio? The read/write ratio narrows what to investigate. Read-heavy workloads make caches, replicas, denormalization, and precomputation worth evaluating. Write-heavy workloads make write contention, batching, partitioning, append-oriented storage, and asynchronous processing worth evaluating. A write-heavy system may still be fine on one database, append-only, fsync-limited, contention-bound, naturally partitionable, batchable — or latency-sensitive and unsuitable for queuing.

3. How much data, and how fast does it grow? Decides whether a naive single-machine layout is still a safe default. The hard distributed-systems work starts when growth, retention, or access patterns outrun that layout — the next mechanism is not automatic.

Here is the arithmetic, made concrete:

"""Back-of-envelope sizing: the numbers that pick your architecture."""

SECONDS_PER_DAY = 86_400


def capacity(dau: int, actions_per_user: int, read_write_ratio: int,
             bytes_per_write: int, peak_multiplier: float = 3.0) -> dict:
    """Turn product numbers into engineering numbers.

    `peak_multiplier` matters more than people expect: traffic is never flat,
    and you must survive the peak, not the average.
    """
    writes_per_day = dau * actions_per_user
    write_qps = writes_per_day / SECONDS_PER_DAY
    read_qps = write_qps * read_write_ratio

    return {
        "write_qps_avg": round(write_qps, 1),
        "write_qps_peak": round(write_qps * peak_multiplier, 1),
        "read_qps_peak": round(read_qps * peak_multiplier, 1),
        "storage_per_day_gb": round(writes_per_day * bytes_per_write / 1e9, 2),
        "storage_per_year_tb": round(writes_per_day * 365 * bytes_per_write / 1e12, 2),
    }


# "Design Twitter" — 200M DAU, 2 tweets/day each, 100:1 read:write, 300 B/tweet
if __name__ == "__main__":
    for key, value in capacity(200_000_000, 2, 100, 300).items():
        print(f"{key:22} {value}")

Running it prints:

write_qps_avg          4629.6
write_qps_peak         13888.9
read_qps_peak          1388888.9
storage_per_day_gb     120.0
storage_per_year_tb    43.8

A runnable version, with Little's Law pool sizing and tail amplification, lives in examples/python/capacity.py.

Now the architecture is no longer a matter of taste. 1.4M application reads/sec is high enough that a conventional single-primary read path should not be assumed to work. Benchmark representative queries and expect to evaluate caching, replicas, denormalization, precomputation, or partitioning — the right mix depends on query complexity, row width, index hit rate, cacheability, working-set size, memory, concurrency, hardware, consistency, network, and result size. 13.9K writes/sec is high enough that write contention, durability settings, and storage layout need an explicit plan; that plan might still be one well-tuned primary, batching, native partitioning, or sharding. 43.8 TB/year means a naive single-disk layout is no longer a safe design. The next decision depends on access patterns, retention, durability, and query behavior and may involve partitioning, multiple tablespaces/storage devices, retention, archival, hot/cold tiering, object storage, separation of metadata and blobs/events, distributed storage, or sharding. You did not choose those pressures; the numbers did — but the numbers name a bottleneck, not a mechanism.

Order-of-Magnitude Anchors — Benchmark the Workload¶

You cannot estimate without a few anchors. These are interview-estimation anchors, not database limits. Real capacity changes by orders of magnitude depending on transaction shape, concurrency, indexes, storage, durability settings, contention, and hardware. Benchmark the actual workload before a number here becomes an architectural decision.

The single most useful consequence: memory is ~1,000× faster than SSD, and a cross-region hop costs more than 1,000 memory accesses. That is why caches exist, and why chatty microservices across regions are a design smell.

What "Good" Looks Like¶

A strong candidate's first five minutes contain almost no architecture:

1. Clarify the actual product. "Is this read-heavy? Who are the users? What is out of scope?"
2. Name the constraint that matters. Latency budget, consistency requirement, or cost ceiling — pick the one that will drive trade-offs.
3. Estimate. Out loud, with round numbers.
4. State the bottleneck. "At 1.4M reads/sec, the conventional read path is under pressure, so the design is really about the read path" — not "the database can't do this," which asserts a mechanism before benchmarking.
5. Then draw.

Steps 1–4 are what separates senior from mid-level. Most candidates skip straight to 5.

Pages in This Section¶

Read them in that order. Requirements-estimation gives you the arithmetic and an interactive calculator, stateless-vs-stateful gives you the vocabulary the rest of the curriculum assumes when it talks about session affinity and horizontal scaling, API Design gives you the contract layer (REST verbs, idempotency, GraphQL, gateways) that every retry/resilience pattern elsewhere in this academy silently depends on, the framework gives you the checklist, and math gives you the queueing theory behind why systems fall over at 80% utilization rather than 100%.

Self-Test Questions¶

Think you know system design? Try answering these without Googling — expand each for a reference answer.

Key Takeaways¶

• Requirements → numbers → bottleneck → architecture. Never skip a step, never reverse the order.
• You need three numbers: QPS, read:write ratio, and data volume + growth.
• Design for peak, not average. Use a 3× multiplier as a floor.
• Memory is ~1,000× faster than SSD; cross-region is ~150 ms. Most caching and colocation decisions follow directly.
• Every box needs a bottleneck to justify it. If you cannot name the pressure, you have not earned the component.

Exit: given a vague prompt, you ask the right questions, estimate scale, draw V1, and name the first bottleneck without naming Kafka.