Stream-table duality

Stream-table duality - stream table duality is the observation that a stream and a table are the same data in two poses: replay a stream of changes and you get a table, watch a table change and you get a stream. Kleppmann calls it turning the database inside out, because every database already works this way in private: the replication log is a stream, each replica is a table built by folding that stream, and the table at any moment is just the log up to that point. Kafka Streams made both poses first-class, a KTable being nothing more than the latest value per key of a changelog topic.

Once you see it, half this glossary compresses. Log compaction is keeping the table half of a stream. A materialized view is a table a query keeps folded from a stream. An upsert changelog is a table travelling as a stream.

Do you need it? Yes, and it costs one sentence to learn. It is also how ubik emits: a windowed aggregate leaves as a changelog keyed by group and window, a table travelling as a stream, so the duality here does real work, not a diagram on a slide.