Stateless vs Stateful Applications¶

Prerequisites: Requirements & Estimation

← Requirements & Estimation | Next: API Design →

Why This Exists¶

"Can we just add another server?" The honest answer is always "it depends where the state lives," and most candidates never say that sentence out loud — they jump straight to "yes, horizontal scaling." If a user's session lives in that one server's memory, adding a tenth server does nothing for the nine users already pinned to servers one through nine, and it actively breaks the one whose next request gets load-balanced somewhere that's never heard of them.

This is not a niche distinction — it is the single decision that determines whether horizontal scaling is a config change or a redesign. Every load balancer algorithm, every sharding scheme, and half of the trade-offs in the trade-off matrix assume you already know which one you're building.

How Each One Actually Works¶

sequenceDiagram
    participant C as Client
    participant S1 as Server A
    participant S2 as Server B

    rect rgb(30, 90, 40)
    Note over C,S2: Stateless — any server, any time
    C->>S1: Request 1 (full context: token, params)
    S1-->>C: Response
    C->>S2: Request 2 (full context again)
    Note right of S2: Server B never talked to Server A.<br/>Doesn't matter — it didn't need to.
    S2-->>C: Response
    end

sequenceDiagram
    participant C as Client
    participant S1 as Server A (holds session)

    rect rgb(120, 40, 40)
    Note over C,S1: Stateful — pinned to one server
    C->>S1: Request 1 — login
    Note right of S1: Session created IN-PROCESS
    S1-->>C: Response + session cookie
    C->>S1: Request 2 (same server, required)
    Note right of S1: Reads session from local memory
    S1-->>C: Response
    end

The stateless server didn't do anything clever to survive Request 2 landing on a different instance — it just never depended on Request 1 having happened anywhere in particular. The stateful server's correctness depends on Request 2 arriving at the same physical process that handled Request 1.

Key Differences¶

Advantages and Disadvantages¶

Stateless

• Highly scalable — load balancers can use simple round-robin or least-connections with no session-affinity constraint
• Better fault tolerance — any instance can serve any request; a dead instance loses nothing durable
• Easier caching — a stateless response for the same input is cacheable at the CDN/edge without worrying about whose session it belongs to
• Cost: more data must travel in each request (a JWT on every call vs. a small session-id cookie); slightly more bandwidth

Stateful

• Simpler client — the client just sends a session ID; the server does the remembering
• Useful for genuinely stateful workflows: a multi-step checkout, a live collaborative document, a WebSocket-based game
• Cost: harder to scale (session pinning or replication required), memory consumption grows with concurrent sessions, and a single server failure can drop live sessions unless mitigated

The Scaling Consequence, Concretely¶

This is the part the interview is actually testing — not the definitions, but what breaks when you scale a stateful design naively.

flowchart TB
    subgraph Naive["Naive stateful scaling — breaks"]
        LB1{{Load Balancer}} -->|round robin| A1["Server A<br/>session: user_42"]
        LB1 -->|round robin| A2["Server B<br/>no idea who user_42 is"]
        U1["User 42, request 2"] -.routed here by chance.-> A2
        style A2 fill:#b71c1c,color:#fff
    end
    subgraph Fixed["Fixed — two options"]
        direction TB
        LB2{{Load Balancer}} -->|sticky by session cookie| B1["Server A<br/>session: user_42"]
        LB2 -.always routes user 42 here.-> B1
        Store[("External session store<br/>Redis / DB")] -.any server reads/writes.-> B2["Server B"]
        LB2 -->|any server, stateless| B2
    end

Two ways to fix a stateful design at scale, and they trade off differently:

1. Sticky sessions (session affinity) — the load balancer routes the same client to the same backend every time, usually via a cookie. Cheap to add, but it caps your scalability to "however evenly sticky routing happens to distribute," and a server failure still drops every session pinned to it.
2. Externalize the state — move session data out of process memory into a shared store (Redis, a database) that every server can read. This is the honest fix: it makes the application servers stateless again, and pushes the actual state-management problem to a system built for it — which is exactly the distributed KV store or cache tier problem, with its own replication and consistency trade-offs.

Token-Based Auth: Stateless by Design¶

JWTs are the canonical example of engineering statelessness into something that used to be inherently stateful (login sessions). The token itself carries the claims (user_id, roles, exp) signed by the server — any instance can verify it with the shared signing key, without a shared session store at all.

This table is itself the trade-off: JWTs buy you stateless horizontal scaling and cost you easy revocation. A system that needs instant "log this user out everywhere, right now" (a compromised account) either accepts a short token TTL, or reintroduces a small piece of state (a revocation list) — there is no free lunch, only a different place to put the complexity.

When to Use What¶

Use stateless when:

• You need to scale horizontally on demand (autoscaling, bursty traffic)
• You're building public APIs or microservices other teams/services call
• You want high availability — no single instance is special
• Token-based auth (JWT) fits your revocation requirements
• You want to leverage CDN/edge caching

Use stateful when:

• The workflow is genuinely session-oriented and short-lived (a multi-step wizard, a live game session, an interactive terminal/WebSocket)
• You're managing a bounded, known number of concurrent users where pinning is operationally acceptable
• The cost of externalizing state (extra network hop to Redis on every request) isn't worth it for your latency budget

Trade-offs¶

Interview Questions¶

Key Takeaways¶

Previous: Requirements & Estimation | Next: API Design