Watermark Simulator

🌊 Why the minimum, not the average

A downstream operator reading from multiple source splits (Kafka partitions, files, shards) cannot fire a window until it is certain no earlier event can still arrive on any of them. So it advances its own watermark to the minimum watermark reported by all splits it considers active — one slow or stalled split holds the whole window back, no matter how far ahead the others race.

• On a single active split, a bounded-out-of-orderness watermark only ever increases: it tracks max event time seen − allowed lateness. A late/old event on that same split does not move its watermark backward.
• An idle split (no new records, e.g. a low-traffic Kafka partition) still reports its last watermark forever. With no idleness policy, the downstream watermark freezes at that value and windows never close — this is the exact failure mode in Flink windows and reproduced by the lab script labs/flink/stalled_watermark.py.
• Marking a split idle (WatermarkStrategy.withIdleness(timeout)) tells the downstream operator to exclude it from the min-computation once it has been silent past the timeout — trading a small correctness risk (events that split resumes emitting may now arrive "late") for progress.
• This is a planning model, not a live cluster: real idleness detection is time-based against wall-clock gaps between records, and re-including a split that starts emitting again involves its own subtleties not modeled here.