Quick Start
Two tools, two short tours. jvq fails the pull request that makes your code slower. jvqq runs jq filters over big JSON Lines files on every core. Each stands on its own.
jvq: Gate a Pull Request
This takes a Go repository with benchmarks from nothing to a pull request check. You need git and Go. Get jvq from the download page, or with Go:
go install github.com/just-very-quickly/jvq/cmd/jvq@latest
Run It Once on Your Laptop
Commit your change on a branch, then from the repository:
jvq run -- go test -run='^$' -bench=. -benchmem ./...
jvq finds where your branch left main, checks out both commits into temporary worktrees, and runs the benchmarks on each, taking turns. Here it is on a small log parser, where a change made the level lookup upper-case the whole line first:
jvq 0.1.0: base a9ec0be920 (merge-base(main, head)) vs head 87815936b0 (HEAD), 12 rounds, 95% intervals
BENCHMARK METRIC BASE HEAD CHANGE INTERVAL BUDGET VERDICT
BenchmarkFields time 205 ns 194 ns -0.6% -7.5% … +4.5% 5% pass
BenchmarkFields bytes 128 B 128 B 0.0% off info
BenchmarkFields allocs 1 1 0.0% off info
BenchmarkLevel time 197 ns 514 ns +158.0% +151.0% … +168.8% 5% REGRESSION
BenchmarkLevel bytes 128 B 224 B +75.0% off info
BenchmarkLevel allocs 1 2 +100.0% off info
verdict: REGRESSION (1 regression, 1 pass, 4 info) in 1m26.8s
How to read it:
- Change is how much slower head is, taken as the median over every pair of base and head runs.
- Interval is where the true change lies, with 95% confidence. The verdict comes from the interval, not from the change alone.
- BenchmarkFields moved by -0.6%, but the interval runs from -7.5% to +4.5%. All of it is within the 5% budget, so it passes.
- BenchmarkLevel is at least 151% slower. Even the kindest reading is over budget: regression, and jvq exits with code 1.
- Bytes and allocations are reported here but not gated, because no budget was set for them.
It ran 12 rounds a side instead of 10. After 10, one result was still undecided, so jvq added rounds until it wasn’t.
Make It Faster
That run took a minute and a half because go test compiled the package in every round. Build the test binary once per side and run that instead:
# jvq.toml, in the repository root
build = "go test -c -o bench.test ./parse"
command = "./bench.test -test.run=NONE -test.bench=. -test.benchmem"
Now jvq run needs no arguments. The worktrees are thrown away afterwards, so the binary doesn’t end up in your checkout.
Not Go?
jvq works the same for any language. Time the whole command:
jvq run -adapter time -- python3 bench.py
Or have the benchmark print its own timings, one JSON line each, and read those:
jvq run -adapter json -- node bench.js
{"name": "parse", "unit": "ns", "value": 912.3}
Put adapter = "json" in jvq.toml to make it the default. The reference has the format in full.
Set the Budgets
[budget]
time = "5%" # the default
allocs = "0" # any new allocation per op fails
[budget.bench."BenchmarkLevel*"]
time = "2%" # tighter on the hot path
jvq check prints the settings it read, so a typo shows up before CI does.
Add It to Pull Requests
# .github/workflows/benchmarks.yml
name: benchmarks
on: [pull_request]
jobs:
jvq:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: actions/setup-go@v7
with:
go-version: stable
- uses: just-very-quickly/jvq@v0.9.0
fetch-depth: 0 matters: jvq needs the base commit, and the default checkout only has the newest one. The action downloads the jvq binary for the runner, checks it against the release’s checksums and runs jvq run with your jvq.toml. The table goes to the job summary, and a regression fails the check with an annotation naming the benchmark.
Or in GitLab
include:
- remote: https://raw.githubusercontent.com/just-very-quickly/jvq/v0.9.0/ci/gitlab/jvq.yml
benchmarks:
extends: .jvq
image: golang:1.26
A regression fails the job and shows in the merge request’s test summary. The reference has the variables.
When It Says Inconclusive
That means the interval still holds the budget after 30 rounds a side: the runner is too noisy to tell whether the change is over budget or not. The run passes, with a note. If that happens a lot, loosen the budget, run more rounds (max_rounds), or narrow the command to the benchmarks that matter. If you’d rather block, set fail_on_inconclusive = true.
The reference has every setting.
Keep Flaky Tests From Blocking
jvq also gates test results. Save your test run’s results and hand them to jvq flaky with a history file:
go test -json ./... > go-test.json
jvq flaky -history jvq-tests.jsonl go-test.json
Any runner’s JUnit XML works too, and several files count as one run. A test that passed and failed on the same commit, twice, goes into quarantine: its failures are still listed, and they stop failing the build. Thirty passing runs in a row take it out again, and a test that fails three attempts in a row blocks no matter what. The reference has the rules.
In GitHub Actions, the action keeps the history in the cache for you:
- run: go test -json ./... > go-test.json || true
- uses: just-very-quickly/jvq@v0.9.0
with:
flaky: go-test.json
Pushes to your default branch add to the history; pull requests are judged against it.
jvqq: Query a Big Log
Get jvqq from the download page, or with Go 1.24 or later:
go install github.com/just-very-quickly/query/cmd/jvqq@latest
Run a Filter
jvqq takes jq’s command line: a filter, then files, or standard input when there are none.
jvqq -c 'select(.level == "error") | {ts, service, msg}' app.jsonl
jvqq -r '.msg' app.jsonl.gz
kubectl logs deploy/api | jvqq -c 'select(.status >= 500)'
The output is what jq would print, in the same order, while every core works on its own part of the file. Files compressed with gzip or zstd are read as they are. The options are jq’s (-c, -r, -n, -s, --arg, --argjson and the rest), plus --workers n to limit the cores.
Count and Group
Instead of reduce inputs or -s, use an aggregation. It runs on every core and holds only the running totals in memory:
jvqq 'select(.level == "error") | count' app.jsonl
jvqq 'count_by(.service)' app.jsonl
jvqq -c 'group_by(.service; {n: count, ms: avg(.latency_ms)}) | .[] | select(.n > 1000)' app.jsonl
The steps before the aggregation run on every line, in parallel. The steps after it run once, on the result.
When a Line Is Broken
jq stops at the first line that isn’t valid JSON. jvqq skips it, keeps going, and says at the end how many it skipped and where the first one was. Add --strict to stop instead, with exit code 2.
The jvqq page has the benchmarks and the full list of differences from jq. The reference has every option.