Out-of-order events

Out-of-order events - what every out of order events streaming discussion is actually about: records arriving in a different order than they happened. This is the normal case. A Kafka partition preserves order within itself and promises nothing across partitions; add a second producer, a retry, a mobile phone that spent an hour in a tunnel, and the arrival order of your stream is a fiction the network composed for you.

The mistake is treating disorder as a failure to engineer away. It is a property of distributed sources, and the field's answer is not to sort the stream but to stop caring about arrival order: window on event time, let each row land in the window its own timestamp names, and let the watermark decide when a window is complete.

Do you need it? You need to accept it, which is free. Once windows are event-time windows, order of arrival stops being your problem: Ubik folds a row into the window its timestamp names whenever it turns up, and only the watermark decides when that window closes. The shuffled stream and the sorted stream give the same answer.