releases
The flaky test gate in 0.7 worked, but setting it up in GitHub Actions meant copying fifteen lines of YAML: restore a cache, pick the right flag for pull requests, run …
releases
The flaky test gate keeps one line per CI run, and through 0.8 that line spelled out every test’s name. Fine for a few hundred tests. For a suite of 2000 Go tests …
guides
jvq flaky needs two things from your CI: test results it can read, and a history file that survives from one run to the next. Everything else is optional. These are the …
releases
Every team has a test like this. It checks out a cart, or uploads a file, or waits on a socket, and about one run in ten it fails for no reason anyone can find. The pull …
case studies
The fastest way to find out what a flaky test gate does with real failures is to give it some. jvq has its own CI history on GitHub: 26 pushes to main between October 18 …
case studies
Logs written by real services aren’t always valid JSON Lines. A process killed mid-write leaves half a line. The restarted process appends its first event straight …
testing
The first round of tests ran jvq hundreds of times on a Go benchmark whose cost we set by hand. That showed how often it raises a false alarm and how often it catches a …
case studies
The alert says checkout is slow. You have the last six hours of application logs as one JSON Lines file, 1 GiB and 3,073,492 lines, and a handful of questions to answer …
releases
A budget gate has a blind spot, and it’s built in. jvq judges each change against its base. A change that makes a benchmark 1% slower passes a 5% budget, and it …
releases
Time is a noisy thing to gate on. On our 2-core test machine the same benchmark spreads 7 to 9% from run to run, so jvq’s default budget is 5%, and a 2% slowdown …