Skip to content

Lab: Postgres Replication

Prediction checkpoint

You will pause the replica, write to the primary, and read from both. Predict the visible values and the lag signal before each command. Then decide whether the application could safely serve that read from the replica.

Pairs with: Replication

A real primary + 2 streaming replicas, cloned via pg_basebackup. Flip synchronous/asynchronous replication live, watch a synchronous write hang when its standby is down, and cause a real split-brain by running pg_promote() — the naive failover Replication warns about.

docker-compose.yml

name: labs-pg-replication

services:
  primary:
    image: postgres:16
    environment:
      POSTGRES_USER: appuser
      POSTGRES_PASSWORD: app_pass
      POSTGRES_DB: appdb
    # Starts fully async. The README walks you through flipping replica1 to
    # a synchronous standby live (ALTER SYSTEM + reload) once it's caught
    # up — turning it on before any standby exists would deadlock the
    # primary's own startup (it would wait forever for an ACK from a
    # standby that can't exist until the primary finishes starting).
    command: >
      postgres
      -c wal_level=replica
      -c max_wal_senders=10
    volumes:
      - ./init-primary.sh:/docker-entrypoint-initdb.d/init-primary.sh:ro
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "appuser"]
      interval: 5s
      timeout: 5s
      retries: 20

  replica1:
    image: postgres:16
    hostname: replica1
    depends_on:
      primary:
        condition: service_healthy
    environment:
      PGDATA: /var/lib/postgresql/data
      PRIMARY_HOST: primary
      APP_NAME: replica1
    entrypoint: ["/replica-entrypoint.sh"]
    volumes:
      - ./replica-entrypoint.sh:/replica-entrypoint.sh:ro
    ports:
      - "5433:5432"

  replica2:
    image: postgres:16
    hostname: replica2
    depends_on:
      primary:
        condition: service_healthy
    environment:
      PGDATA: /var/lib/postgresql/data
      PRIMARY_HOST: primary
      APP_NAME: replica2
    entrypoint: ["/replica-entrypoint.sh"]
    volumes:
      - ./replica-entrypoint.sh:/replica-entrypoint.sh:ro
    ports:
      - "5434:5432"

replica-entrypoint.sh

The replica's custom entrypoint — clones from the primary via pg_basebackup on first boot, then hands off to the standard postgres entrypoint:

#!/bin/bash
# Custom entrypoint for the replicas: clone from the primary via
# pg_basebackup on first boot (empty data dir), then hand off to the
# standard postgres entrypoint. -R writes standby.signal and
# primary_conninfo for us — that's what makes this a streaming replica.
set -e

if [ -z "$(ls -A "$PGDATA" 2>/dev/null)" ]; then
  echo "[$APP_NAME] data dir empty — waiting for primary, then cloning"
  until pg_isready -h "$PRIMARY_HOST" -U repl_user -d postgres; do sleep 1; done
  pg_basebackup -h "$PRIMARY_HOST" -U repl_user -D "$PGDATA" -Fp -Xs -P -R \
    -d "application_name=${APP_NAME}"
  chmod 700 "$PGDATA"
fi

exec docker-entrypoint.sh postgres

Exercises

The full walkthrough (confirm replication, flip sync/async live, watch a sync write hang, promote a replica and observe the resulting split-brain) lives in the lab's README:

labs/postgres-replication/README.md on GitHub

git clone https://github.com/sanketn26/interview-prep
cd interview-prep/labs/postgres-replication
docker compose up -d

← All Labs