DynamoDB Deep Dive: Managed Serverless Scale¶

Partition key: user_id

AWS hashes the partition key to decide which physical partition an
item lives on. The number of partitions is NOT fixed — DynamoDB adds
partitions automatically as a table grows past capacity or storage
thresholds for its current partition count, and splits an existing
partition when it outgrows its own throughput ceiling.

What IS fixed, and load-bearing for capacity planning: each partition
has a hard throughput ceiling (~3,000 RCU or 1,000 WCU, whichever is
hit first) and a storage ceiling (~10GB). Once a table needs more
capacity than one partition can serve, DynamoDB adds partitions and
redistributes data across them — you don't control or see this
directly, but the ceiling per partition is why a single hot key can
throttle even when the table overall has huge configured capacity.

AWS replicates each partition across multiple Availability Zones for
durability — the replication factor itself isn't a number DynamoDB
exposes as a tunable, unlike Cassandra's RF.

Partition key: user_id
Sort key: timestamp (DESC)

Query: "Get all events for user_id='alice' in last 7 days"

WHERE user_id = 'alice' AND timestamp > now() - 7 days
→ Scans partition for alice, filters by timestamp
→ Fast (if sorted order matches query)

Main table (user_id partition):
  Allocated: 1000 RCU (read capacity units), 500 WCU (write capacity units)

GSI (email partition):
  Allocated: 1000 RCU, 500 WCU (separate from main table!)

Total cost = main table + all GSIs

Write a user:
  Main table: 1 WCU
  GSI-1 (email): 1 WCU
  GSI-2 (created_at_day + created_at): 1 WCU
  → 3 WCU total

Table: 1 billion users
Attribute: phone (only 10% have it)

Without sparse index:
  GSI stores 1 billion items (even if 900M have null)

With sparse index:
  GSI stores 100 million items (only those with phone)
  Much smaller, cheaper

Allocated: 1000 RCU, 500 WCU per second

Cost: ~$0.09 per RCU-month + ~$0.47 per WCU-month
      (approximate us-east-1 list prices; dated — check current)

1000 RCU: ~$90/month
500 WCU: ~$235/month
Total: ~$325/month

If you exceed capacity:
  → Requests are throttled (HTTP 400 ProvisionedThroughputExceededException)
  → Application must retry

Cost: ~$0.25 per million RCU + ~$1.25 per million WCU

1 million reads: $0.25
1 million writes: $1.25

If traffic spikes:
  → Capacity auto-scales to absorb it, cost increases with usage
  → BUT on-demand does NOT mean throttling is impossible:
    - Per-partition throughput still has a hard ceiling (a single
      partition tops out around 3,000 RCU / 1,000 WCU) — a hot key
      can still get throttled even on-demand, same failure as
      provisioned mode, see Part 5
    - On-demand tables also cap how fast they scale: a table can
      generally handle up to double its previous peak traffic
      immediately, but a much larger, sudden spike can still outrun
      that ramp-up window and throttle until capacity catches up
    - Account-level and table-level service limits still apply

Item: user_id="alice", balance=100

UPDATE users SET balance = 50 WHERE user_id="alice";

DynamoDB Stream:
  MODIFY: {
    "user_id": {"S": "alice"},
    "balance": {"N": "100"}        -- old value
  }
  →
  {
    "user_id": {"S": "alice"},
    "balance": {"N": "50"}         -- new value
  }

DynamoDB table → stream → Lambda function
                          (e.g., update cache, send email, log analytics)

1. Write to DynamoDB
2. Stream captures change
3. Lambda reads stream, indexes in ES
4. ES stays in sync with DynamoDB

Partition key: user_id

Celebrity user (100M followers):
  Every write to user_id="elon" goes to one partition
  That partition maxes out write capacity
  Other partitions sit idle

Actual: 1 partition at 95% capacity while others at 5%

Instead of: user_id="elon"
Write to: "elon#" + random(0, 100)

"elon#0": write 1
"elon#1": write 1
...
"elon#99": write 1

Total: 100 writes distributed across 100 partitions

Read: Query "elon#*" (all 100 keys)

If elon's profile is read 10M times/sec:
  Cache in ElastiCache (Redis)
  Only cache misses hit DynamoDB

AWS DynamoDB's adaptive capacity:
  Reallocates a table's UNUSED provisioned throughput toward a hot
  partition, up to that partition's own hard ceiling (~3,000 RCU /
  1,000 WCU per partition).

What it does NOT do:
  - Does not replicate or split the hot partition itself
  - Does not raise the per-partition throughput ceiling
  - Cannot help once the hot partition is already saturating that
    ceiling — a single truly hot key (like "elon") can still throttle
    even with adaptive capacity fully engaged, because the ceiling is
    per-partition, not per-table

It buys you some headroom when a partition is hot but the table still
has slack capacity elsewhere — it does not make a single key infinitely
scalable. Solution 1 (spread the key itself across multiple physical
partitions) is still the actual fix for a genuinely hot single key.

BEGIN TRANSACTION
  UPDATE accounts SET balance = 100 WHERE user_id="alice";
  UPDATE accounts SET balance = 50 WHERE user_id="bob";
COMMIT;

If either update fails → entire transaction rolls back
Both succeed → both changes persist

Normal write: 1 WCU
Transactional write: 2 WCU

If you do 1000 transactional writes/sec:
  Provisioned: 2000 WCU (vs 1000 for normal writes)
  Cost: 2× higher

Backup automatically (up to 35 days retention)

Restore: restore_time="2024-08-10 03:00:00"
→ DynamoDB creates a new table with data from that point
→ No downtime for original table

CREATE TABLE sessions (
  session_id string PRIMARY KEY,
  created_at timestamp,
  ttl_timestamp timestamp
);

Set TTL on column: ttl_timestamp

INSERT sessions VALUES ('session_123', now(), now() + 1 hour);
-- After 1 hour, item is automatically deleted (no write cost)

DynamoDB table in us-east-1
  ↕ (replicates both ways)
DynamoDB table in eu-west-1

Write in us-east-1 → automatically replicated to eu-west-1 (< 1 second)
Read from nearest region → always fast