Point it at a Kafka topic, write SQL, get results that survive a kill -9. It runs windowed aggregations as one process on one machine - no JobManager, no RocksDB tuning, no three SREs on call - . If your streaming job fits on a laptop, and most do, you never needed the cluster.
Closed source. Free to use, on a couple of conditions.
Everyone deploys a cluster - Flink on Kubernetes, a JobManager, TaskManagers, RocksDB tuning, three SREs - to run jobs that would fit on two cores of a laptop. Most production Flink jobs process under 100k events/s. The cluster is cargo cult; we talked ourselves into Netflix's infrastructure depth for a workload Netflix would run on a cron job.
DuckDB already made this argument for analytics: collapse the whole thing into one binary, go where the data already lives, and it turns out "small to mid" is almost everyone. Ubik does the same to real-time. And no, nothing here reinvents hot water - the SQL is DuckDB's, the Kafka client is librdkafka's - . What's ours is the part that did not exist yet; the operators, the watermarks, the checkpoint that comes back byte-identical after a kill -9.
Point it at a live topic, hand it one SQL string, watch windowed aggregates roll into your terminal. The same job in Flink is a docker-compose with four services and a Maven build you will get to know intimately.
$ ubik --from kafka://localhost:9092/txns \
--to kafka://localhost:9092/txn_counts \
--follow --checkpoint ./txns.ckpt \
"SELECT merchant,
TUMBLE(event_time, INTERVAL 1 MINUTE) AS window,
count(*) AS n
FROM txns
GROUP BY merchant, TUMBLE(event_time, INTERVAL 1 MINUTE)"Kill it mid-window. Resume from the checkpoint. The counters pick up exactly where they were, and when you run DuckDB batch over the same data cold, the numbers match to the cent. That comparison runs in CI on every merge - exactly-once you can watch, not a line in a datasheet - .
$ # SIGKILL mid-stream, then resume from the checkpoint
$ ubik --from kafka://localhost:9092/txns \
--to kafka://localhost:9092/txn_counts \
--resume ./txns.ckpt \
"SELECT ... GROUP BY ..."
# resumed run is byte-identical to an uninterrupted oneIf it is on this list, there is a test proving it. If it is not, we do not claim it. See what is not here further down.
DuckDB frontend, full scalar surface. Three streaming extensions (TUMBLE, HOP, WATERMARK), same shape as Flink SQL. Nobody learns a DSL to sum counters per window.
TUMBLE, GROUP BY, HAVING, count/sum/avg/min/max. The watermark closes windows and emits mid-stream. Late events are counted and reported, never dropped in silence.
Operator state and Kafka offsets in one atomic checkpoint. A resumed run is byte-identical to an uninterrupted one, checked against a DuckDB oracle in CI.
Deterministic replay from the checkpoint plus an idempotent upsert key. A crash mid-window is a first-class test case, not an incident you get paged for.
Aggregate state spills to disk past a memory budget, so state larger than RAM does not OOM. Hundreds of GB on NVMe, on one box.
Source and sink over the Kafka protocol. Works against the Kafka, Redpanda, WarpStream or MSK you already run. No "first, migrate your events".
SCRAM, PLAIN, OAUTHBEARER, mTLS via vendored static OpenSSL. Credentials stay in the environment, never written to config, logs, state or checkpoints.
ubik pipeline create / start / stop / list / logs. One detached process per pipeline, resumed on restart. The OS is the supervisor, not some control plane.
No cluster, no JobManager, no RocksDB tuning. The binary is the deploy: one process, one state file, links no external shared libraries.
It is closed source, and it is free to use - on a couple of conditions we are still writing down - . It is pre-alpha too, so "getting it" means we hand you a build and you tell us the moment it breaks. Leave an email and we will set you up.
Once it is on your machine, the whole thing is one command. Run SQL over a file, results as JSON lines on stdout - the table name is the source basename - .
$ ubik --from file://events.ndjson \
"SELECT merchant, amount FROM events WHERE amount > 100"
{"merchant":"acme","amount":250.0}
{"merchant":"acme","amount":980.0}When you want it durable, name the pipeline and let the CLI supervise it. One detached process, resumed from the last checkpoint across a restart.
$ ubik pipeline create txns \
--from kafka://localhost:9092/txns \
--to kafka://localhost:9092/txn_counts \
"SELECT ... GROUP BY ..."
$ ubik pipeline start txns
$ ubik pipeline list
NAME STATUS PID FROM TOAbove a few million events/s sustained, state beyond one machine’s NVMe, or hot-state sub-second failover; take Flink, and the docs will say so without flinching.
It reads and writes logs, it never becomes the log of record. You bring the durable Kafka-shaped thing upstream. Rebuilding one is scope creep we are not interested in.
Output goes to Kafka, stdout, or a sink. The charts-and-dashboards half of the family lives at rawquery.dev, the sister product.
No distributed code path will ever land in the core. The moment it learns about a second machine, we have rebuilt Flink and lost the plot.
Mono-node is the whole point, so Ubik Cloud stays the same unchanged binaries with an orchestration layer bolted on top. Hosted pipelines, supervised, no box for you to babysit at 11pm. Same engine, same exactly-once, none of the ops.
Now, I will be honest - I have not fully drawn the shape yet - . Leave an email and I will reach out when there is something real to run.