Optimize model building pipeline: defer Dependency.build() and reduce allocations - #12653
Optimize model building pipeline: defer Dependency.build() and reduce allocations#12653gnodet wants to merge 1 commit into
Conversation
46712b6 to
5e74cd4
Compare
Benchmark Results (4383-module project,
|
| Version | Average | vs rc-6 |
|---|---|---|
| rc-6 baseline | 22,114ms | — |
| PR #12652 (pool+sort) | 18,617ms | -15.8% |
| PR #12653 (before this commit) | 17,739ms | -19.8% |
| PR #12653 (with this commit) | 17,956ms | -18.8% |
Performance is within run-to-run variance. The fix eliminates the unnecessary full-model materializations while maintaining the same performance level. All 1130 tests pass (550 in maven-impl, 580 in maven-core).
Updated benchmark: getModifiable() + two-field optimizationFollowing up on the previous fix — the pipeline stages now use What changed (commit 443dcc5)model.vm (two-field optimization):
Pipeline stages (getModifiable pattern):
Cost modelBefore (v1): each pipeline stage that touches deps does a full wrap + unwrap cycle: With 3 stages touching deps: 6N allocations per module (3 wraps + 3 unwraps) After (v2): ONE lazy wrap + ONE final unwrap for the entire pipeline segment: Total: 2N allocations per module (1 wrap + 1 unwrap) Benchmark results (4383-module project,
|
| Build | Average | vs rc-6 | vs PR #12652 |
|---|---|---|---|
| rc-6 baseline | 20,234ms | — | — |
| PR #12652 (pool+sort) | 14,936ms | -26.2% | — |
| PR #12653 v1 (getBuilt) | 14,784ms | -26.9% | -1.0% |
| PR #12653 v2 (getModifiable) | 13,385ms | -33.8% | -10.4% |
The v2 optimization delivers a clear 10% improvement over PR #12652 alone, and 34% over rc-6.
Branch: optimize-model-builder-pipeline-fix (commits 568b44b861 + 443dcc5b51)
JFR-guided optimization (v3):
|
| Component | v2 samples | v2 % | v3 samples | v3 % | Change |
|---|---|---|---|---|---|
| DependencyMgmtImporter | 442 | 43.0% | 143 | 20.4% | -68% |
↳ updateWithImportedFrom |
354 | 34.4% | 44 | 6.3% | -88% |
| CopyOnWriteArraySet | 68 | 6.6% | 0 | 0.0% | -100% |
| ModelValidator | 71 | 6.9% | 66 | 9.4% | same |
| I/O (XML, Zip, Jar) | 78 | 7.6% | 86 | 12.3% | same |
| Other | 369 | 35.9% | 406 | 57.9% | same |
| Total CPU samples | 1028 | 701 | -32% |
What was fixed
1. updateWithImportedFrom (was 35% of CPU)
Every BOM-imported dependency was rebuilt through Dependency.newBuilder(dep, true).importedFrom(loc).build() — triggering forceCopy (wraps all sub-lists into builders), build (unwraps back to immutable), and object pool interning — just to set one metadata field.
Added withImportedFrom(InputLocation) + private copy constructor to model.vm. Shares all model fields by reference, bypasses Builder and object pool entirely.
2. CopyOnWriteArraySet (was 6.6% of CPU)
ConsumerPomArtifactTransformer.deferDeleteFile() added to a CopyOnWriteArraySet<Path> → O(n) indexOfRange scan + array copy per add. With 4383 modules: O(n²). Replaced with ConcurrentHashMap.newKeySet() → O(1).
Wall-clock benchmark (4383 modules, mvn validate)
Wall-clock difference between v2 and v3 is within noise (~0.3s on a 14s benchmark) because:
- BOM import runs once on the root POM, not on the critical path for parallel child processing
- The parallel 4383-module build is I/O and scheduling dominated
The cumulative improvement from rc-6 baseline remains ~27% faster.
New commit: Store single model-object fields as sub-buildersAdded commit What changedmodel.vm — single model-object fields now use sub-builders: Builder classes now store fields like
Pipeline stages updated to use
ImpactEach pipeline stage that modifies
On a 4383-module reactor, each module passes through ~7 pipeline stages. Previously, every stage that touched 🤖 Generated with Claude Code |
625b0b0 to
20afa1a
Compare
Store list and single-object fields as sub-builders in generated Builder classes so the pipeline can thread a Model.Builder end-to-end without materialising intermediate immutable objects. Key changes: - Builder list fields store Collection<X.Builder> internally; getters lazily build immutable snapshots only when read - Single-object fields (e.g. Build, DependencyManagement) stored as sub-builders with getModifiable*() accessors for in-place mutation - Add visitBuilder() to MavenTransformer and builder-accepting default methods on SPI interfaces (ModelNormalizer, ProfileInjector, etc.) - Thread Model.Builder through readEffectiveModel pipeline stages to avoid build()/newBuilder() round-trips between stages - Eliminate unnecessary builder.build() calls in pipeline stages that only pass the model to the next stage Benchmark on 4384-POM reactor (mvn validate, 10 runs): rc6: avg 52.8s median 52.4s this: avg 31.4s median 32.3s → 40% faster Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
e7827b5 to
9949734
Compare
Summary
model.vm) — enables reading builder fields through the base chain without callingbuild(), supporting deferred materialization across pipeline stages*ToBuildermerger variants (merger.vm) — returnsBuilderinstead of callingbuild(), letting callers accumulate changes across multiple merge passes without intermediate allocationsbuild()inDefaultDependencyManagementInjector— accumulates merged dependencies asBuilderobjects, callingbuild()only once per modified dependency at the end of the loopcomputeLocations()— replacesStream.concat().collect(Collectors.toUnmodifiableMap(...))withHashMap.putAll()+Map.copyOf(), and returnsoldlocsdirectly whennewlocsis empty (avoids unnecessaryMap.copyOfsince the base map is already immutable)locationsHashCode— cached at build time in the generated constructor, used byDefaultModelObjectPool.PoolKeyfor fast inequality checks before full map comparisonPoolKey.locationsEqual()— uses directgetLocations()map comparison instead of iterating individual keys, with fast-path for both-empty maps (common for imported deps)addLocationInformationAPI toXmlReaderRequest— wired throughDefaultModelXmlFactoryfor future use in skipping location tracking on imported BOM POMsContext
JFR profiling on a 4,383-module reactor shows 37% CPU in
ModelObjectPool(dependency interning), 18% inDependencyimmutable builders, and 2.3 GB allocated inKeyValueHolderfromcomputeLocations(). EachDependencypasses through ~7 pipeline stages, each callingbuild()which allocates a new immutable object + recomputes location maps + callsprocessObject().This PR targets the three hottest code paths:
build()calls — deferred via Builder getters and ToBuilder merger variantscomputeLocations()stream overhead — replaced with HashMap merge + early returnPoolKeylocation comparison — precomputed hash + direct map equalityTest plan
mvn verify -Bpasses across all reactor modules (all tests green)FileToRawModelMergerTestupdated to exclude new*ToBuildermethods from reflection-based override check🤖 Generated with Claude Code