JUnit Publication and Test Trends

Scenario: Tests fail but Jenkins receives only a shell exit code, so regressions and flaky cases remain invisible.

Jenkins can't read your test runner's mind

Here's a fact that trips people up constantly: if your test runner exits with code 0 but never actually generates a structured report, Jenkins has genuinely no way to know that 40 real test assertions ran and passed - all it can see is "the shell command didn't crash." Turning real test output into something Jenkins can track, trend, and report on requires a specific, structured format.

Analogy: Imagine a teacher who only gets to see whether a student's homework folder was submitted on time (a shell exit code), with no way to see the actual grades on individual problems inside unless the student also fills out a standardized grade-report form (a JUnit XML report). "Submitted on time" and "actually did well" are completely different facts, and without the standardized form, the teacher genuinely cannot tell them apart - one student who left every answer blank but submitted an empty folder on time looks identical, from the outside, to one who correctly solved every problem, unless someone fills in that report.

This is exactly why the junit step exists: it's the standardized "report card" format that lets Jenkins actually see and track what your tests found, instead of just whether the test command happened to exit cleanly.

Pipeline example

post { always { junit testResults: 'reports/**/*.xml', allowEmptyResults: false } }

Structured reports create real test history

The junit step parses compatible XML report files and attaches individual suites, cases, durations, failures, skips, and historical trend data to the build. Jenkins genuinely cannot infer individual test cases from a shell command's exit code alone - it needs the actual structured report. Configure your test runner to write out reports even when tests fail (some runners skip report generation on a crash by default, which is exactly the wrong behavior for CI), and publish those reports in post { always { ... } } so a failing run still gets its evidence recorded. Use a precise glob pattern, and generally reject empty results (allowEmptyResults: false) so a broken test runner or a wrong working directory can't quietly present itself as a clean, successful, zero-tests-found run.

[Pipeline] junit
Recording test results
Test results: 118 tests, 116 passed, 2 failed, 0 skipped
Trend: 2 new failures since build #211
Finished: UNSTABLE

The "Trend" line is exactly what real, structured reporting buys you over a bare shell exit code - Jenkins can tell you not just "2 tests failed" but "these 2 failures are new since the last passing build," which is enormously more useful when triaging what actually changed.

Define what unstable actually permits downstream

Test failures commonly make a build unstable rather than an outright failure; malformed report XML or genuinely missing required report files, by contrast, can cause the junit step itself to fail the build outright. Decide deliberately, as policy, what an unstable result is allowed to do downstream: diagnostic collection and informational dashboards may reasonably continue on unstable, while promotion to production should generally require a real, unambiguous success. Preserve enough console context around a failure while still keeping test names stable across runs - a test that gets renamed between commits looks, to Jenkins' trend tracking, like the old test was silently removed and an unrelated new test was silently added, breaking the continuity of its failure history right when you'd want to trace it back further.

Flakiness is a defect signal, not an annoyance to suppress

Automatically retrying a test suite until it happens to go green biases your recorded history toward looking healthier than reality. Retain the result of the first attempt specifically, clearly identify which runs were retries, and measure intermittent failure rate broken down by individual test and by environment (a test that's flaky only on one OS is a different, more specific problem than one that's flaky everywhere). Quarantine a genuinely flaky test only with a named owner, a documented reason, and an explicit expiry date - a "temporary" quarantine that nobody revisits is really just silent, permanent deletion of that test's coverage, dressed up to look like a deliberate decision.

A pitfall: missing reports that look like "everything passed"

For missing test reports, methodically check: did the test runner actually exit in a way that still wrote the report file, is the working directory what you assumed it was, do timestamps suggest the report is stale from a previous run rather than freshly generated, are file permissions correct, and - especially relevant after the matrix and parallel modules - could two parallel branches be colliding on the same report output path and overwriting each other. For one specific test that suddenly regressed, compare its duration and failure history across recent commits and across different matrix cells, rather than looking at the current failure in isolation. Jenkins' trend data becomes genuinely useful only once "report is completely absent," "build aborted due to infrastructure," and "a real assertion actually failed" are kept distinct from each other - conflating any two of these into one generic "red build" erases exactly the information you'd need to triage quickly.

Goal: Practice the concrete behavior in jenkins-labeled-parallel-tests. Learn sections have no Jenkins terminal; the Jenkins lab opens the real shared service in a new full-screen tab inside your private folder.