Watermark

Watermark - the standard answer to what is a watermark in stream processing: the engine's moving claim that it has now seen every event up to time T, so any window ending before T can close and emit. It is how a system that never stops decides that something is finished.

Now, the honest part. A watermark is a bet on punctuality, not a fact. The engine watches how far along in event time each source has got, takes the minimum, and calls that progress. The Dataflow paper allows for a perfect watermark; real ones are heuristics, and when the bet loses, the straggler is late data. That is the whole concept. One moving timestamp, occasionally wrong on purpose, because waiting forever is the only way to be certain and nobody ships "forever" as a latency target.

Do you need it? The concept, absolutely: it is the idea behind every closed window. The machinery, less so. The big engines give you watermark strategies, generators and idleness policies to configure. In Ubik the watermark is computed for you as the minimum event time across the source's partitions, and the slack is one flag, --watermark-delay. The concept survives. The config files do not.