Observability

What a run reports on stderr, the final stats line and the live progress line, the resource knobs, and the license notice.

A run's results go to stdout or a sink. Everything about the run itself goes to stderr as JSON lines: a status, a final stats, a periodic progress, and a license notice. An in-process host reads the same facts through typed calls instead; see ubik_progress and ubik_stats.

The stats line

Every run ends with one stats line, its summary.

{"stats":{"rows_in":8400,"rows_out":12,"late_dropped":0,"windows_closed":6,"rows_folded":8400}}

Field	Meaning
`rows_in`	Source rows read.
`rows_out`	Result rows emitted.
`rows_folded`	Rows folded into aggregate state.
`windows_closed`	Windows that closed and emitted.
`late_dropped`	Rows dropped because their window had already closed. A rising number means the source is more out of order than `--watermark-delay` allows.
`spilled_blocks`	State blocks spilled to disk under memory pressure.

The line also carries peak_resident_blocks and workers, which depend on the machine, not the data.

The progress line

--progress-interval emits a progress snapshot every interval, so a long run is not silent between start and the final stats.

ubik --from file://big.ndjson "<windowed SQL>" --progress-interval '5 seconds'

{"progress":{"rows_in":245760,"rows_out":1500,"ev_per_sec":4915200,"peak_ev_per_sec":4915200,"watermark":"2026-07-18 00:30:00","checkpoint":null,"late_dropped":0,"spilled_blocks":0,"lag":null,"workers":16,"windows_closed":30,"rows_folded":245760}}

It is a live snapshot, so the counters are wherever the run was at the tick, not a fixed result. ev_per_sec is the current throughput, watermark is the coordinated event-time watermark, checkpoint is the last checkpoint's sequence and age (null until one commits), and lag is the Kafka consumer lag (null until measured, or on a file source). At a terminal the cadence drops to 500ms so the display stays live; 0 seconds disables it.

Resource knobs

Set through the environment, or the matching ubik_options field in process.

Variable	Controls	Default
`UBIK_WORKERS`	Parallel probe degree for aggregate and join.	All hardware threads
`UBIK_STATE_MEMORY_MB`	Resident aggregate-state budget before spilling.	1024
`UBIK_SPILL_DIR`	Where spilled state blocks go.	`$TMPDIR`, else `/tmp`
`UBIK_HOME`	Root for pipeline state and logs.	`$HOME/.ubik`

Pipeline logs

A pipeline captures its engine's stdout and stderr to $UBIK_HOME/pipelines/<name>/engine.log as JSON lines, the same status, stats and progress a foreground run prints. ubik pipeline logs <name> -f follows it. The log is size-capped and rotated so a long-running pipeline does not fill the disk.

For a pipeline under systemd, the auto-resume --pipeline form plus Restart=on-failure is the whole recovery story; the unit is on the exactly-once page.

The license notice

Every run prints one license line. Unlicensed, it reads:

{"license":{"state":"unlicensed","notice":"Unlicensed: free for personal, non-commercial and evaluation use. Commercial use needs a license.","url":"https://getubik.dev/#pricing"}}

It reports, it does not gate. Ubik runs the full engine at full capacity in every state; a key silences the notice and unlocks nothing. An in-process host reads the state through ubik_license and renders it in its own idiom.