Labs¶
The Playgrounds simulate a mechanism in the browser, fast and safely. These are the next step: the same failure, on a real Kafka broker, a real Postgres replica, a real Kubernetes cluster — with the timing, edge cases, and outright bugs a JS model doesn't bother simulating.
Each lab's page here shows its actual docker-compose.yml (or Terraform/Kubernetes config) inline — always in sync with the real file, since the page pulls it in at build time rather than copy-pasting it. The step-by-step exercises are longer and live in the lab's own README.md in the repo's labs/ directory, linked from each page below. Every lab in this list has actually been run end-to-end while building it, not just written.
Requirements¶
- Docker and Docker Compose v2 (
docker compose versionshould work) kubernetes-kindadditionally needskindandkubectl(brew install kind kubectl)terraform-dockeradditionally needs Terraform (brew install hashicorp/tap/terraform)- A few GB of free RAM if you run more than one lab at a time
- Nothing here is meant to stay running — every lab ends with
docker compose down -v(orkind delete cluster/terraform destroy)
Labs¶
| Lab | Pairs with | What you can actually do that the simulator can't show |
|---|---|---|
| Kafka cluster | Kafka Deep Dive | Kill a real broker mid-traffic, watch a real leader election and a real consumer-group rebalance |
| Postgres replication | Replication | Flip sync/async live, watch a sync write hang when its standby is down, cause a real split-brain with pg_promote() |
| Redis Sentinel | Replication | Watch a 3-node quorum vote a new master in after killing the old one, and watch it reconfigure the old master as a replica automatically |
| Sharded Postgres (Citus) | Sharding | Query pg_dist_shard_placement to see real shard balance, watch a cross-shard query plan fan out, reproduce a real hot shard |
| etcd cluster | Raft, CAP Theorem | Kill a minority (writes keep working) vs. a majority (writes refuse, not corrupt) of a real Raft quorum; watch --consistency=serializable succeed without quorum while the default (linearizable) fails |
| Rate limiter races | Rate Limiting | Reproduce a real TOCTOU race that lets 20 requests through a limit of 5, and a real "TTL never expires" bug — then fix both with one atomic Lua script |
| Retry storm | Circuit Breakers | Inject a real fault with Toxiproxy and measure retry amplification directly: 10 client requests become ~40 real backend hits |
| Load balancer algorithms | Load Balancing | Swap round robin / weighted / least-connections on a real nginx upstream block and watch the distribution actually change |
| Kubernetes (kind) | Kubernetes | A real 3-node cluster: break a Service selector, break a readiness probe, diagnose both with real kubectl |
| Terraform + Docker | Infrastructure as Code, Terraform | Real plan/apply/destroy against real containers (no cloud account): prove idempotency, trigger real drift detection, watch a real -/+ forced replacement |
How to use one¶
Then follow that lab's README.md — a numbered sequence of exercises. Tear down when done; nothing here is designed to be left running.
Lab cycle (every exercise): predict → run → compare → explain.
- Predict the log line, error, or metric before you run the command.
- Run it.
- Compare the actual output to the prediction.
- Explain any gap in your own words; if you were surprised, re-read the paired concept page before the next step.
This is the same discipline the Playgrounds page asks for, applied to a real process.
A note on what these are for¶
None of these are production-representative deployments — they use trust auth, no TLS, no resource limits, and configurations chosen for clarity over security or performance. Do not copy these compose files into anything that isn't a throwaway local lab. The point is to make one specific mechanism (leader election, quorum failover, shard placement, retry amplification) observable in minutes, not to teach production operations end to end.