Requirements & Capacity Estimation¶

Prerequisites: HTTP basics, System Design Framework (pair this with steps 3–5)

← Foundations | Next: Stateless vs Stateful →

Why This Exists¶

"Design WhatsApp." Two candidates draw the same boxes. Only one can say:

2B users, 50% DAU → 1B DAU
40 msgs/user/day  → 40B msgs/day
40B / 86,400      → ~460k avg msg/s
peak 5–10×        → 2–5M msg/s
100 B text        → 40 TB/day raw
RF=3 + media      → this is a storage company that also chats

Without those numbers you cannot choose Postgres vs a log, one region vs three, or a 4-core box vs a fleet. Estimation is not accounting — it is which physics applies.

Naive System → What Breaks¶

You hear "10 million users" and size a single 16-core API + one Postgres primary.

The naive system dies at connections and tails, not at "we ran out of CPU."

The Concept¶

Functional requirements are what the product does. Non-functional requirements (NFRs) are how wrong / how slow / how often that is allowed.

WhatsApp-shaped example:

If you list "highly available and fast" you have not written NFRs. Write numbers.

Architecture¶

flowchart LR
    DAU[DAU] -->|× actions/day| DAY[Reqs / day]
    DAY -->|/ 86400| AVG[Avg QPS]
    AVG -->|× peak| PEAK[Peak QPS]
    PEAK -->|read %| RW[Read QPS / Write QPS]
    RW -->|× payload| BW[Bandwidth]
    RW -->|write × 86400 × bytes| ST[Storage / day]
    ST -->|× 365 × RF × regions| YR[Stored / year]
    RW -->|× hit %| CACHE[Cache vs origin]
    PEAK -->|× conns/req| CONN[LB / pool / FD]
    style PEAK fill:#e65100,color:#fff
    style YR fill:#b71c1c,color:#fff
    style CACHE fill:#1565c0,color:#fff

Mechanics¶

Seconds in a day = 86,400. Memorize 10^5 ≈ day. N / 10^5 is avg QPS within 15%.

10M DAU × 20 req/user/day = 200M req/day
200M / 86,400             ≈ 2,315 avg QPS
peak multiplier 8–10×     ≈ 18–23k peak QPS
read 90%                  ≈ 21k read / 2.3k write at peak
payload 2 KB              ≈ 23k × 2 KB ≈ 46 MB/s peak

Reads vs writes. Writes hit the source of truth and the WAL. Reads hit cache, then replicas. A 90/10 split is a cache problem. A 10/90 split (ingest, telemetry) is a log problem.

Payload. Control plane (JSON 1–2 KB) and data plane (photos, video chunks) must be estimated separately. One 1 MB avatar upload at 1% of users/day swamps the 2 KB chat ACKs.

Bandwidth both ways. Response often larger than request (feeds). Count QPS × (in + out).

Storage.

writes/day × avg_bytes × 365 × RF × regions × (1 + index overhead ~0.3)

Do not forget indexes, WAL, snapshots, and "deleted" rows you still keep for 30 days.

Replication. RF=3 is three copies plus repair traffic. Cross-region RF is a latency and conflict decision, not just a multiplier.

Cache. Origin QPS ≈ read_QPS × (1 − hit_rate). 80% hit on 21k read QPS → 4.2k origin. 50% hit → 10.5k. The DB you buy is the miss fleet.

Connections.

peak QPS × latency_s = in-flight   (Little's Law)
in-flight / pods = conns per pod

20k QPS × 0.08s = 1,600 in-flight. 20 pods → 80 conns each — fine. p99 = 2s under GC → 40k in-flight → pool wait → death spiral. See Tail Latency.

Realistic Example With Numbers¶

Feed service: 10M DAU, 20 req/user/day, peak ×8, 90% read, 2 KB, 1 region, RF=3, 80% cache hit.

Daily reqs          200,000,000
Avg QPS             2,315
Peak QPS            18,519
Peak read / write   16,667 / 1,852
Cache hits          13,333 QPS
Origin reads        3,333 QPS
Peak NIC (in+out)   ~74 MB/s  (2 KB each way)
Avg write QPS       2,315 × 10% ≈ 232 write/s   (peak write QPS is 1,852)
Writes / day        232 write/s × 86,400 × 2 KB ≈ 40 GB/day   (use average, not peak)
Year logical        40 GB × 365 ≈ 14.6 TB
With RF=3           ~44 TB  (+indexes/WAL → budget ~60 TB)
In-flight @ 80ms    18.5k × 0.08 ≈ 1,480

Estimates, not accounting. Being within 2–3× is a pass. Being off by 100× (forgot peak, or stored every read) is a fail.

Interactive Explainer¶

Change DAU, peak, payload, RF, and hit rate. Flags fire when a naive single-primary layout is no longer a safe default — they name a bottleneck, not a mandatory mechanism.

Failure Modes¶

Production Debugging¶

Capacity mistakes show up as saturation, not as a wrong spreadsheet.

CPU         high + p99 high     → you undersized compute; scale out
CPU         low  + p99 high     → not a capacity miss — tail, lock, dep
Memory      OOM / cache evict   → working set > RAM; hit rate lie
Disk        WAL / log fill      → writes/day × RF you forgot
Network     NIC 80% + retrans   → payload × QPS; jumbo / compression
Queue depth Kafka / SQS climb   → consumer QPS < peak write QPS
Lag         seconds growing     → same; partitions or slow handler
Pools       wait > 10ms         → in-flight = QPS × latency; add pods or cut p99
p50/p95/p99 p50 ok p99 5s       → see tail latency; averages hide this
Error rate  503/429             → limiter or overload; you hit a real limit
Timeouts    climbing with load  → pool + retry amplification
Retries     downstream RPS 3×   → your estimate used client QPS, not amplified
GC          pause == p99        → heap sized for peak live set, not avg
Locks       row / global        → write QPS concentrated on one key

Scaling Limits¶

• A single 16-core app pod: ~5–15k simple QPS. 20k peak ⇒ more than one pod before Kafka.
• One Postgres primary: often quoted in the low-single-digit k durable writes/s as an interview-estimation anchor, not a limit — real capacity depends on the workload; sharding or a log is not the automatic next step.
• One Redis shard: 50–150k simple GET/s; hot key is the real limit, not the average.
• NIC: 1–2 GB/s before you think "this is a CDN / blob problem."
• File descriptors and conntrack fill before CPU at chat-scale connection counts.
• Multi-region RF turns every write into a WAN RTT unless you accept eventual.

Trade-offs¶

Interview Questions¶

Reasoning Exercises¶

1. WhatsApp: 1B DAU, 40 msgs/user/day, 80% text (100 B), 20% media (200 KB). Compute daily storage and peak msg/s. Which component do you scale first?
2. Your cache hit rate drops from 90% to 60% after a deploy (key namespace change). Using the calculator defaults, what happens to origin QPS and to the primary?
3. A trading venue: 2k avg QPS, peak 80× at open. Why is "size for 2k" malpractice? What NFR do you write instead?
4. 10k long-lived WebSockets per box, 200k users online. How many boxes for connections only? How does that differ from the QPS fleet?

Key Takeaways¶

Previous: Foundations | Next: Stateless vs Stateful