Scripted Pipeline and CPS

Scenario: A legacy workflow creates stages dynamically but retains non-serializable objects across durable Pipeline steps.

Why does Jenkins sometimes "pause" a running build?

Here's a Jenkins behavior that genuinely surprises newcomers: Jenkins can restart entirely - a server reboot, an upgrade - and a Pipeline that was in the middle of running can pick up again afterward, right where it left off, instead of starting over from scratch. That "pause and resume" ability is not magic; it comes at a real cost, and Scripted Pipeline is where that cost becomes most visible.

Analogy: Imagine writing a letter by hand, but every few sentences you have to physically hand the paper to someone else to hold onto, in case you need to step away entirely and someone else has to continue writing later from that exact page. You could write "the customer said their favorite color is blue" (a plain fact, easy to write down and hand off) - but you couldn't hand off "I'm currently pointing my finger at the fourth word in this paragraph while thinking about it" (a live mental pointer, meaningless once separated from your hand). Jenkins' Scripted Pipeline works the same way: at certain checkpoints, it needs to write down the entire current state of the program as if pausing to hand it off - and only simple, "just the facts" values like text, numbers, and lists survive that handoff cleanly. A live "pointer" object, like an in-progress file iterator, does not.

This mechanism is called CPS (continuation-passing style), and understanding why it exists explains a whole category of confusing Scripted Pipeline errors.

Pipeline example

node('linux') { try { stage('Test') { sh './test.sh' } } finally { junit 'reports/*.xml' } }

CPS changes ordinary Groovy assumptions

Scripted Pipeline uses node, stage, parallel, try/finally, and generally normal-looking Groovy control flow. Jenkins transforms this code with continuation-passing style (CPS) so that execution can genuinely pause at durable steps (like sh, sleep, or input) and resume later, even across a full controller restart. To make that possible, any state that's "alive" across a suspension point must be serializable - convertible into a plain, storable form and back. Iterators, streams, regex Matcher objects, and live Jenkins model objects held across an sh, sleep, input, or other durable step frequently cause a NotSerializableException, or worse, silently unsafe replay behavior if the serialization technically "works" but doesn't mean what you expect after a resume.

What that error actually looks like

java.io.NotSerializableException: java.util.regex.Matcher
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup...
Finished: FAILURE

This error is Jenkins telling you, quite literally: "I tried to write down the current state of your program so I could pause here, and one of the objects you're holding onto (a regex Matcher, in this case) can't be written down." The fix is almost never "catch this exception" - it's restructuring the code so that only plain data (strings, numbers, lists, maps) is held across the durable step that triggered the pause.

Keep Groovy as orchestration, not as the workload

Convert data to plain strings, numbers, lists, and maps before it needs to cross a durable step. @NonCPS is appropriate only for pure computation that calls no Pipeline steps and returns a serializable value - it is not a universal escape hatch for "code that's giving me CPS trouble." CPU-heavy Groovy logic and traversal of Jenkins' internal API objects run on the controller itself, even inside what conceptually feels like "the build" - so genuinely heavy work (compiling, running tests, processing large data) should be pushed out to sh steps running real tools on an agent, not left as raw Groovy computation.

Dynamic stages: powerful, but easy to make unreadable

Generating stages dynamically in a loop is justified when the actual set of things to test or deploy is only known at runtime (like "one stage per file changed in this commit"). But when the dimensions are already known in advance - like "test on Node 18 and Node 20" - that's clearer expressed as a Declarative matrix or a set of explicitly named parallel branches, both of which show up cleanly in Jenkins' visual pipeline graph, unlike a dynamically-built loop. Give generated branches stable, meaningful names, cap how many can run at once, and publish reports inside a finally block so a failure partway through the loop doesn't silently skip evidence collection. When closures are created inside a loop, capture the loop variable into a local value first (def name = branchName) - otherwise every generated branch can end up sharing the same final value of the loop variable, a classic and very confusing closure bug.

A pitfall: diagnosing a resume that went wrong

To diagnose a resume failure, locate the last durable step that completed, inspect exactly what value was being held across it, review whether any plugins changed versions recently (plugin updates can change what's serializable), and reduce the problem to the smallest possible reproducing example. Deliberately test a controller restart while a Pipeline is sleeping, waiting on input, and mid-way through an agent step, so you find out how your specific Pipeline behaves before a real production restart forces you to find out the hard way. A well-designed Scripted Pipeline stays understandable in the stage view, survives the suspension points it's actually likely to hit, and never turns the Jenkins controller itself into a general-purpose application runtime.

Goal: Practice the concrete behavior in jenkins-labeled-parallel-tests. Learn sections have no Jenkins terminal; the Jenkins lab opens the real shared service in a new full-screen tab inside your private folder.