Mostly Harmless!FeaturesEmbedCloudOEMLive shellExactly-onceBenchmarksAlternativesDocsPricing
Mostly HarmlessDelivery & correctness

Deterministic replay

Deterministic replay is what recovery leans on: the same input re-run yields the same output.

Also called: replayability

KAFKAdurable logcheckpointed offsetubikone processCHECKPOINTone atomic unitsource offsetsoperator statekill -9SINKsame bytes after resumew0w1w2resume: rewind to the checkpointed offsets, replay
The checkpoint is one atomic write: the source offsets and the operator state land together, or not at all. kill -9 at any instant; resume reads the unit, rewinds Kafka to the recorded offsets, replays, and the sink carries the same bytes, nothing lost, nothing doubled. A stream-stream join rides the same unit: both source offsets, both watermark clocks, the two stores and the emit cursor. Proven byte-identical to the DuckDB batch oracle across kill -9 mid-window and mid-band.

Deterministic replay - re-run the same input through the same computation and get the same output, which is the deterministic replay streaming engines rest their whole recovery story on. If the log is replayable and the computation is deterministic, a crash stops being an emergency: load the last checkpoint, replay the log from the recorded offset, arrive at the state you would have had anyway. Kreps's log essay made the first half canon; the second half is on you. Wall-clock timestamps, random numbers, iteration order over a hash map, a lookup table that changed underneath: each one makes a replay quietly diverge from the run it replaces.

So determinism is a discipline. Event time instead of processing time, seeded randomness or none, pinned reference data.

Do you need it? Yes, and it is the cheapest correctness property on this page: no coordinator, no protocol, just refusing nondeterminism. Ubik is built on it, replay from one atomic checkpoint, and a resume refuses to run if the already-consumed log or a lookup dimension changed since that checkpoint was taken. Recovery is boring, which was the goal.

Sources

Related

Reprocessing and backfillExactly-onceCheckpoint
← All 50 terms