Cloud Load Balancers: NLB, ALB, and Multi-Cloud¶

Prerequisites: Load Balancing, Proxies & Ingress

← Networking Overview

Why This Exists¶

You have 1M QPS. You deploy behind AWS ALB. It works for a day, then starts dropping requests. Why? You hit ALB's throughput limit, not because the code is slow, but because the load balancer itself became the bottleneck.

This page teaches the layer, throughput, and cost differences so you pick the right load balancer for your use case — not the one Marketing says is "simpler."

The OSI Stack Matters¶

Layer 4 (NLB)¶

What it sees: TCP/UDP packets, source/dest IP, port, seq numbers.

What it can do: - Route by port number - Route by IP protocol - Very fast (millions of packets/sec) - Cannot read HTTP headers (doesn't know what /api is) - Cannot cache responses

Use case: Databases, gaming servers, extreme throughput, non-HTTP protocols.

Request: TCP packet to 203.0.113.1:443
NLB: "Port 443? Route to backend pool 1"
(NLB doesn't care if it's HTTPS or SSL or something else)

Layer 7 (ALB)¶

What it sees: Full HTTP request (headers, body, path, query string).

What it can do: - Route by HTTP path (/api → api-servers, /static → cdn) - Route by hostname (api.example.com → api-servers, web.example.com → web-servers) - Route by HTTP method (POST → api-servers, GET → cache) - Route by HTTP header values - Cache responses - Slower than L4 (must parse HTTP) - Suitable for 10k–100k RPS per instance

Use case: Microservices, REST APIs, websites, anything HTTP.

Request: GET /api/users HTTP/1.1
ALB: "Path is /api? Route to api-servers"

AWS Load Balancers¶

AWS Classic LB (ELB) — Don't Use¶

Old (2009). Supports L4 and basic L7. Deprecated in favor of ALB/NLB.

AWS Network Load Balancer (NLB)¶

For: Ultra-high throughput, low latency, non-HTTP protocols.

Throughput: Millions of requests per second.
Latency: ~100µs (microseconds).
Cost: $0.006 per LCU-hour + $0.006 per GB processed.

Capabilities: - L4 routing (TCP, UDP, TLS) - Source IP affinity (sticky sessions via IP hash) - Connection draining (gradual shutdown) - Cross-zone load balancing (automatic, slight latency cost)

When to use: - Gaming servers (UDP, extreme latency-sensitive) - Databases (TCP, high concurrency) - IoT endpoints (MQTT, custom protocols) - Extremely high throughput (>100k RPS) where ALB becomes bottleneck

When NOT to use: - HTTP APIs (use ALB instead, cheaper and simpler) - Request-based routing (cannot read HTTP headers)

Example: Databases behind NLB — do not hash primary+replicas together

Hashing a writer and its replicas behind one NLB sends writes to a replica. Postgres replicas reject or silently diverge on writes.

Wrong: one NLB pool {primary, replica, replica} — hash may land writes on a replica

Right:
  Writer endpoint (NLB or DNS) → primary only
  Reader endpoint              → replicas (and maybe primary, if you want)
  Or a role-aware proxy (PgBouncer / ProxySQL / RDS Proxy / Cloud SQL Auth Proxy)

AWS Application Load Balancer (ALB)¶

For: HTTP/HTTPS, REST APIs, microservices, websites.

Throughput: 10k–100k RPS (depending on request size).
Latency: ~10–20ms.
Cost: $0.0225 per LCU-hour + $0.006 per GB processed.

Capabilities: - L7 routing (HTTP methods, paths, hostnames, headers) - Host-based routing (api.example.com → api-servers) - Path-based routing (/api/v1/* → api-servers, /static/* → cache) - Query string routing - HTTP header routing - Request prioritization - Cookie-based sticky sessions

When to use: - REST APIs (99% of microservices use this) - Multiple services behind one load balancer - Request-based routing logic - HTTP caching

When NOT to use: - Extreme throughput (>100k RPS) — use NLB - Non-HTTP protocols (databases, gaming) — use NLB - Extreme latency sensitivity (ALB adds ~10ms) — use NLB

Example: Microservices behind ALB

ALB
├─ Rule: api.shop.com/api → api-service:8080
├─ Rule: api.shop.com/auth → auth-service:8080
├─ Rule: web.shop.com → web-service:3000
├─ Rule: cdn.shop.com → cdn-service:8080
└─ Default → 404

Client: GET https://api.shop.com/api/users
ALB: "Host is api.shop.com, path is /api → api-service"
ALB: "Route to api-service:8080"

AWS Gateway Load Balancer (GWLB)¶

For: Third-party appliances (firewalls, proxies, intrusion detection).

How it works: Transparently inserts appliances into the traffic path without changing routing.

Client → GWLB → [Appliance 1, 2, 3] → Backend

Appliance can inspect/modify traffic (firewall rules, logging).

GCP Load Balancers¶

GCP Cloud Load Balancer (Internal)¶

For: L4 ultra-high throughput (Maglev algorithm).

Throughput: Millions of packets/sec (internal).
Latency: ~1µs (sub-millisecond).
Cost: $0.02 per hour + $0.01 per GB.

Why it's fast: Maglev algorithm (consistent hashing) distributes traffic across load balancer instances.

GCP Cloud Load Balancer (External, L7)¶

For: Global HTTPS load balancing.

Throughput: 1M+ RPS (auto-scales).
Latency: Depends on geography (traffic goes to nearest edge).
Cost: $0.025 per hour + $0.02 per million requests.

Capabilities: - L7 HTTP(S) routing - Geographic routing (route to nearest data center) - URL path routing - Request-rate based autoscaling - Global load balancing (routes across regions)

Comparison with AWS:

Azure Load Balancer¶

Azure Basic LB¶

For: Simple internal load balancing (similar to AWS NLB but L4 only).

Throughput: 1M–5M flows.
Latency: Low (L4).
Cost: $0.025 per hour.

Capabilities: - L4 TCP/UDP routing - Port forwarding - Outbound NAT rules

Azure Application Gateway¶

For: L7 HTTP(S) load balancing (similar to AWS ALB).

Throughput: 10k–100k RPS.
Latency: ~20ms.
Cost: $0.01 per hour + $0.009 per GB.

Capabilities: - L7 HTTP routing - WAF (Web Application Firewall) - SSL/TLS termination - URL-based routing - Host-based routing - Cookie-based sessions

Azure Front Door¶

For: Global CDN + load balancing (similar to GCP Cloud LB).

Throughput: Unlimited.
Latency: Geographic (routes to nearest edge).
Cost: $0.01 per hour + $0.06 per million requests.

Comparison Table: When to Use What¶

Real-World Pitfalls¶

Pitfall 1: Picking NLB When ALB Suffices¶

Wrong: Microservices behind NLB because "NLB is faster."

Problem: You lose HTTP routing. Every service needs its own NLB = expensive. You pay for ultra-high throughput you don't use.

Right: Use ALB for microservices. If p99 latency is a problem, measure it. ALB adds ~10ms; if your SLO is 100ms, it's fine.

Pitfall 2: Hitting ALB Throughput Limit¶

Scenario: 500k RPS, 1KB requests, behind one ALB.

ALB can handle this (capacity is in LCUs, not request count). But if you misconfigure and the ALB becomes a bottleneck, your SLO breaks before you realize it.

Fix: Monitor ALB metrics (ProcessedBytes, TargetCount, UnHealthyHostCount). Use multiple ALBs or NLB if you outgrow ALB.

Pitfall 3: Cookie-Based Sticky Sessions Breaking¶

sequenceDiagram
    participant U as User
    participant ALB as ALB
    participant S1 as Server-1
    participant S2 as Server-2

    U->>ALB: Request 1 — login
    ALB->>S1: route (no cookie yet)
    S1-->>U: session created, Set-Cookie AWSALB
    U->>ALB: Request 2 (AWSALB cookie)
    ALB->>S1: sticky route to server-1
    S1-->>U: response

    Note over S1: Server-1 crashes

    U->>ALB: Request 3 (AWSALB cookie, still points to server-1)
    ALB->>S1: try server-1
    S1--xALB: unreachable
    ALB->>S2: reroute to server-2 (no session data)
    S2-->>U: "You've been logged out"

Request 1: User logs in, ALB routes to server-1
         Server-1 sets AWSALB cookie
Request 2: User makes another request, ALB sees AWSALB cookie
         Routes to server-1 (sticky)

Server-1 crashes.
Request 3: User makes request, ALB tries to route to server-1 (dead)
         If ALB can't reach server-1 after N seconds, routes to another server
         User sees: "You've been logged out" (because new server doesn't have their session)

Fix: Don't rely on sticky sessions for sessions. Use a session store (Redis) instead.

Pitfall 4: Cross-Zone Load Balancing Cost¶

ALB in us-east-1a sends traffic to backends in us-east-1b
Cost: $0.006 per GB (in addition to normal LCU cost)

If you have 1 TB/day = $6/day = $180/month extra.

Fix: Place backends in the same AZ as the ALB (for cost). Or accept the cost if you need HA across AZs.

Cost Comparison: Real Numbers¶

Scenario: 100k RPS, 1KB average request¶

Math first: 100k RPS × 1 KB × 86,400 × 30 ≈ 259 TB/month of processed bytes, not 260 GB. ALB is billed on LCUs (the max of new connections, active connections, processed bytes, rule evaluations) plus a small hourly ALB charge — not "$0.0225 per LCU-hour" as the only line.

AWS ALB (order of magnitude): processed bytes alone are hundreds of GB/hour → hundreds of LCUs, so thousands of $/month, not ~$18. Quote current LCU prices from AWS; do not treat the hourly ALB fee as the total.

AWS NLB: also LCU-based (bytes + connections + flows). Same traffic is still thousands $/month, usually cheaper than ALB at this volume but not "six dollars."

GCP HTTP(S) LB: request-count + data processing; at 2.6B requests/month this is also thousands, not $70.

Do not use fake ~$18 totals in an interview. The real point: at 100k RPS the LB data/LCU line dominates, and L7 costs more than L4 for the same bytes.

Interview Questions¶

Key Takeaways¶

Previous: Proxies & Ingress