build(test): stop test forks oversubscribing the machine

[jamesfredley]

Draft — opened as a draft so CI measures the change before review. See Measuring this on CI.

The problem

A plain ./gradlew build can leave a developer's workstation unusable. Observed on a 14-core / 20-thread machine: 23 JVMs, 12.4 GB, with the desktop unresponsive.

Three multipliers stack, and each looks reasonable on its own.

1. availableProcessors() counts logical processors. SMT threads on x64, and on Apple silicon every efficiency core as well as every performance core. * 3 / 4 of that is already measured against an inflated number — 15 forks on a 14-core host, and on an M-series Mac most of those forks land on efficiency cores.

2. maxParallelForks is per Test task. With org.gradle.parallel=true, several test-bearing modules run concurrently, so the real ceiling is Gradle's worker-lease pool (--max-workers, default = availableProcessors), not any single task's fork count.

3. Each forked JVM sizes its own thread pools for the whole machine, because no fork knows the others exist. Measured on JDK 21 at 20 visible processors:

Setting	Value
ParallelGCThreads	15
ConcGCThreads	4
CICompilerCount	12
Total per fork	31

That is 31 threads per fork before a single test runs. Gradle cannot see it: it charges each fork one worker lease, as though a fork were a single thread. Gradle's own default for Test.maxParallelForks is 1 for precisely this reason — raising it opts out of the protection Gradle already provides.

15 forks × 31 threads ≈ 465 threads contending for 20 hardware threads, plus 15 × 1 GB heaps (2 GB in grails-test-suite-persistence) against a 5 GB daemon. That is the thrashing.

The change

Applied to all three builds in this repository — root, grails-gradle and grails-forge each have their own settings.gradle, so the duplication is unavoidable.

1. Local test forks: 3/4 of the logical processors → half. CI keeps its existing budget (isCiBuild ? 4, and ? 3 in grails-gradle), so CI fork counts are unchanged.
2. Every test fork is told how many processors it may size its pools from, as availableProcessors / maxWorkerCount, floored at 2. The denominator is the build-wide worker limit rather than any one task's maxParallelForks, because that is what actually bounds how many forks run at once. A floor of 2 keeps G1 rather than dropping to Serial GC.

Supplied via jvmArgumentProviders rather than jvmArgs, because several modules assign jvmArgs wholesale and would discard it.

Also fixes grails-forge, where -PmaxTestParallel was silently ignored.

Measured result

:grails-core:test --rerun-tasks, daemons stopped between runs, comparing second runs:

Order	Baseline	This branch	Change
baseline first	3m 25s	2m 45s	19.5% faster
treatment first	3m 05s	2m 24s	22.2% faster

Running this branch first rules out filesystem-cache ordering bias — the advantage held in both directions.

	Baseline	This branch
Forks	15	10
ActiveProcessorCount	20 (default)	2
Threads per fork	31	5
Projected total threads	~465	~50
Peak JVMs	29	24

Thread figures are a computed projection from java -XX:+PrintFlagsFinal -version, not a live thread count.

Second commit: isolatedTestsTwo was not actually isolated

grails-test-suite-uber/build.gradle set maxParallelForks = 1 on isolatedTestsTwo, but registered that block before the tasks.withType(Test).configureEach block in the same script. Gradle runs both as deferred configuration actions in registration order, so the later configureEach overwrote it and the task ran fully parallel. Its test patterns have been deliberately serialized since 2013 because they are order sensitive.

Moving the override after the configureEach block fixes it. CI impact is negligible: the task filters three classes and sharding assigns it to a single shard.

Measuring this on CI

Baseline for gradle.yml on 8.0.x: median 2h 14m, p90 3h 52m, with core build jobs at 1h 20m – 1h 40m. Draft PRs do trigger the workflow, so this PR's own run is the measurement. Since CI fork counts are unchanged, any CI movement comes from the per-fork processor cap alone.

Follow-ups, deliberately not in this PR

• A memory budget. An earlier revision budgeted forks against physical RAM, but a per-task budget cannot bind build-wide, so it was removed rather than shipped as a guarantee it could not keep. Bounding aggregate memory needs org.gradle.workers.max or a shared BuildService.
• A 5 GB Gradle daemon on a 7 GB macOS runner (GitHub's macOS runners are 3 vCPU / 7 GB, versus 4 vCPU / 16 GB on Linux and Windows) is already marginal and worth revisiting separately.
• Testcontainers multiply with forks. The Mongo, Redis and forked Geb suites hold containers in per-JVM statics with no withReuse, so N forks means N containers — and on macOS and Windows that memory comes from a Docker Desktop VM the JVM cannot see.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.3088%. Comparing base (b964e60) to head (13c9641).

Additional details and impacted files

@@                Coverage Diff                 @@
##                8.0.x     #16158        +/-   ##
==================================================
- Coverage     52.3252%   52.3088%   -0.0164%     
+ Complexity      18536      18529         -7     
==================================================
  Files            2039       2039                
  Lines           97498      97498                
  Branches        17138      17138                
==================================================
- Hits            51016      51000        -16     
- Misses          38998      39011        +13     
- Partials         7484       7487         +3     

see 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[testlens-app]

✅ All tests passed ✅

🏷️ Commit: 13c9641
▶️ Tests: 68812 executed
⚪️ Checks: 79/79 completed

---

Learn more about TestLens at testlens.app.

[jamesfredley]

CI measurement (first draft run)

All 59 jobs passed. Comparing the Build Grails-Core matrix jobs against the median of the three most recent successful gradle.yml runs on 8.0.x (runs 31893711990, 31892777441, 31730582241):

Job	Baseline median	This PR	Change
Ubuntu JDK 21	84.8 min	59.5 min	-29.8%
macOS JDK 21	77.9 min	62.5 min	-19.8%
Windows JDK 25 shard 2	45.4 min	36.6 min	-19.4%
Ubuntu JDK 25	81.0 min	77.4 min	-4.4%
Windows JDK 25 shard 1	43.1 min	43.5 min	+0.9%
Windows JDK 25 shard 0	77.1 min	95.4 min	+23.7%

Whole-workflow wall clock was 95.5 min against a ~2h14m median.

How much to trust this

Not much yet, on its own. This is one run against a three-run baseline, and per-job variance on GitHub runners is large: baseline Windows JDK 25 shard 0 alone ranged 66.8 - 93.7 min across those three runs, so this PR's 95.4 min is only just outside its own baseline spread. Treat the Windows shard 0 regression and the Ubuntu JDK 21 improvement with equal caution until there are more samples.

What makes the direction plausible rather than noise is that the local A/B benchmark was run in both orderings on an idle machine, and the improvement held each way (19.5% with baseline first, 22.2% with this branch first).

What is actually being measured here

CI fork counts are unchanged by this PR - the isCiBuild budgets stay at 4 (root) and 3 (grails-gradle). So any CI movement comes solely from the per-fork -XX:ActiveProcessorCount cap, not from running fewer forks. On a 4-vCPU runner each fork now sizes its GC and JIT pools for 2 processors instead of 4, which is a much smaller change than the local one (20 processors down to 2).

The macOS result is the most interesting one for day-to-day work, since most committers develop on macOS and GitHub's macOS runners are the smallest at 3 vCPU / 7 GB.

Suggested next step

Re-run this workflow a couple more times before drawing conclusions, so each job has a comparable sample count to the baseline.

[jamesfredley]

CI measurement, second sample - correcting the first

A second run of the same commit landed (59/59 jobs green again). It does not reproduce the improvement reported above, and the earlier numbers should be treated as retracted.

Job	8.0.x baseline median	Sample 1	Sample 2	Mean vs baseline
Ubuntu JDK 21	84.8 min	59.5	80.4	-17.5%
Ubuntu JDK 25	81.0 min	77.4	75.5	-5.6%
macOS JDK 21	77.9 min	62.5	89.9	-2.2%
Windows JDK 25 shard 0	77.1 min	95.4	96.6	+24.5%
Windows JDK 25 shard 1	43.1 min	43.5	45.1	+2.8%
Windows JDK 25 shard 2	45.4 min	36.6	44.2	-11.0%

What this actually shows

Runner variance dominates. macOS JDK 21 moved from 62.5 to 89.9 minutes on identical code - a 35 percentage point swing between two runs. Any single-sample CI comparison on this workflow, including my first one, is noise. The honest reading of two samples is that CI wall clock is roughly unchanged.

Windows JDK 25 shard 0 is the one consistent signal, slower in both samples (+23.7%, +25.3%). Two samples is still thin, but it is the only job where both point the same way, so it deserves attention rather than dismissal. Worth noting the baseline for that job spans 66.8-93.7 minutes across three runs, so even this may be variance.

Does this invalidate the change?

Not the local result, which is the stronger evidence and was measured under controlled conditions - idle machine, daemons stopped between runs, and critically run in both orderings so filesystem-cache bias pointed against the change in one of them:

baseline first:   baseline 3m25s, this branch 2m45s   (19.5% faster)
branch first:     this branch 2m24s, baseline 3m05s   (22.2% faster)

That reproducibility is what a shared GitHub runner cannot offer.

It is also worth restating what CI is even exercising here: fork counts are unchanged on CI by design (isCiBuild ? 4), so the only CI-visible effect is the per-fork ActiveProcessorCount cap - 4 processors down to 2 on a runner, versus 20 down to 2 locally. A small or unmeasurable CI effect is the expected outcome, not a contradiction.

The developer-machine problem this PR exists to fix - 23 JVMs, 12.4 GB, and roughly 465 threads on a 20-thread box - is unaffected by any of this.

Suggested next step

If CI timing is a merge criterion, this needs several more samples per job to say anything, particularly for Windows shard 0. If it is not, the local A/B plus unchanged CI fork counts should be sufficient.