Transactional messaging
Kafka transactions group writes across partitions into one atomic unit that consumers see entirely or not at all.
Also called: read_committed
Transactional messaging - Kafka transactions, specified in KIP-98: a producer groups writes to several partitions, plus its consumer offset commit, into one unit that readers see entirely or not at all. The mechanism runs through a transaction coordinator inside the broker. The producer registers a stable transactional id, the coordinator records begin and commit markers in a transaction log, and a consumer set to read_committed skips anything from an aborted or still-open transaction. The stable id also fences zombies: a restarted producer bumps an epoch, so the crashed instance's in-flight transaction is aborted rather than left to commit late.
What it buys is cross-partition atomicity, the piece the idempotent producer cannot give you. What it costs is latency, since read_committed consumers wait for the commit marker, plus coordinator round-trips and a class of operational surprises around transaction timeouts.
Do you need it? To operate yourself, rarely: only when one logical write genuinely spans topics. Ubik uses exactly this machinery at its Kafka sink, one transaction per closed window, so a read_committed consumer sees a window atomically or not at all. You consume the guarantee; you do not run the coordinator.