Reprocessing and backfill

Reprocessing and backfill - the stream reprocessing backfill move: running history back through the same pipeline that handles the live feed, to fix a bug, add a column, or rebuild derived state from scratch. Kreps's point in The Log is that the log already retains the history, so reprocessing is just a consumer starting from an earlier offset. There is no second system to build.

The reason it sounds terrifying is a memory of pipelines that secretly depended on the wall clock. A pipeline built on event time does not care that "now" is wrong during a replay: each historical row lands in the window its own timestamp names, out-of-order arrival included, and the replay computes the same result the live run did. Determinism is the whole trick, and you either have it or you do not.

Do you need it? Yes, and often: backfill is how derived data gets fixed. In Ubik, replay is the default mode, not a feature: point it at a topic or a file without --follow and it runs to the head, closes every window, and stops. The CI harness compares that streamed answer to a DuckDB batch over the same events at rest, byte for byte. Your backfill is the batch answer.