Jvqq on JVQ.net

A Crashed Writer Left Three Broken Lines in a 1 GiB Log and jq Stopped at the First

Mon, 19 Oct 2026 00:00:00 +0000

Logs written by real services aren’t always valid JSON Lines. A process killed mid-write leaves half a line. The restarted process appends its first event straight after it, on the same line. Disk-full conditions, log shippers that split long lines and two writers sharing one file all do something similar. The file is 99.9999% fine and has a few lines that no parser will accept.

jq treats the input as one stream of JSON, so the first bad line ends the run. This worked example shows what that looks like and what jvqq does instead. We took 1 GiB of generated logs, 3,073,492 lines, and cut three lines in half, each followed by the next line on the same line, the way a crashed writer leaves them. In the broken file they’re lines 812,345, 1,604,990 and 2,949,998. The commands below ran with jq 1.7.1 and jvqq 0.1.0 on a two-core cloud VM.

How jvqq Traced a 12-Minute Payments Outage Through 3 Million Log Lines

Mon, 19 Oct 2026 00:00:00 +0000

The alert says checkout is slow. You have the last six hours of application logs as one JSON Lines file, 1 GiB and 3,073,492 lines, and a handful of questions to answer before anyone can fix anything. Which service? Since when? Which endpoint? Each question is a jq filter, and with jq the first one alone took half a minute.

This is a worked example. The log is generated by jvqq’s benchmark generator, and we planted an incident in it: from 03:10 to 03:22, about 60% of the billing service’s payment requests fail with 503 after 2 to 4 seconds. Everything else in the file is ordinary traffic, with its usual background of errors. The commands and output below are real, run with jvqq 0.1.0 and jq 1.7.1 on a two-core cloud VM.

jvqq Reads a Week of Gzipped Logs Faster Than zcat Can Unzip Them

Mon, 19 Oct 2026 00:00:00 +0000

Most log files older than a day are compressed. Logrotate gzips them, object storage keeps them as .gz, and the habit for querying them is a pipe: zcat app-*.jsonl.gz | jq .... It works, and it’s slow twice over. gzip unpacks on one core, and jq parses on one core, each waiting on the other.

jvqq reads gzip and zstd files directly, recognized by their first bytes, and parses on every core. This worked example is a week of rotated application logs: seven daily files from jvqq’s benchmark generator, 150 MiB each uncompressed and about 23 MB each compressed, 161 MB in all and 3,151,367 lines. The commands below ran with jvqq 0.1.0 and jq 1.7.1 on a two-core cloud VM, with the files already in the page cache.

jvqq Runs jq Filters on Every Core and Is Tested Against jq Byte for Byte

Mon, 19 Oct 2026 00:00:00 +0000

Application logs tend to end up in the same shape: a file of a few gigabytes with one JSON event per line, and a question about it. How many 500s per service since the deploy? Which requests hit the timeout? jq answers that kind of question well. On a big file it takes its time, because it reads on one core.

jvqq 0.1 is a command-line tool that takes the same filters and runs them on every core. The output comes out in input order and is what jq would print, down to the byte. On a four-core GitHub-hosted runner, seven filters over 1 GiB of logs took 0.7 to 1.8 seconds each with jvqq and 17 to 35 seconds with jq.