Allowed lateness

Allowed lateness - the grace period a window stays open after the watermark passes its end, a knob most people meet by searching allowed lateness Flink at some point in their first month. The watermark says "probably done"; allowed lateness says "keep the state around a little longer in case it was wrong".

Mechanically it is simple. In Flink, a window's state is retained until the watermark passes end plus the configured lateness; a straggler inside that grace is folded in and the window fires again as an update, which your downstream now has to treat as an update rather than a final answer. After the grace, the row is late data proper: dropped or side-outputted. The default is zero, which is the honest default, because the number is really a statement about how out of order your source is, and nobody knows that better than your source.

Do you need it? The number, yes: one interval, measured against reality, revised when late_dropped says so. In Ubik it is --watermark-delay, which holds each window open until end plus the interval. One flag, default zero, and the stats line tells you when it is too small.