Skip to content

Fix flaky WhereQueryClosureCaptureSpec: key AstPropertyResolveUtils cache by ClassNode identity - #16034

Open
borinquenkid wants to merge 3 commits into
8.0.xfrom
fix/flaky-where-query-closure-capture-cache
Open

Fix flaky WhereQueryClosureCaptureSpec: key AstPropertyResolveUtils cache by ClassNode identity#16034
borinquenkid wants to merge 3 commits into
8.0.xfrom
fix/flaky-where-query-closure-capture-cache

Conversation

@borinquenkid

Copy link
Copy Markdown
Member

Summary

WhereQueryClosureCaptureSpec has two flaky feature methods (~2% flaky per
#16030, 0 hard failures) verifying that closures generated by the
where {} query transform only capture the variables they actually reference.

Root cause

AstPropertyResolveUtils.cachedClassProperties is a static, non-thread-safe HashMap
used by the where-transform's variable-scope recompute path
(DetachedCriteriaTransformer). It was already keyed by ClassNode.getName()
(qualified name), not a bare simple name as first suspected — but ClassNode.equals()/
hashCode() also compare by name, and this spec's own test deliberately compiles
identical source twice into two separate GroovyClassLoaders, producing two
distinct ClassNode instances that are equals()-identical and therefore collide as
the same cache entry. This is reachable by any two classes sharing a name across
separate compilations, not just this test — a prior partial fix (08bd0bf) only
renamed this spec's fixtures to reduce collisions with sibling specs, without fixing
the cache itself.

Fix

  • Switched cachedClassProperties to a synchronized IdentityHashMap<ClassNode, ...>
    — keys compared by ==, so two distinct ClassNodes can never collide regardless of
    shared naming.
  • Fixed a related concurrency bug in the same method: a new cache entry was previously
    published into the shared map before its population loop finished, letting a
    concurrent reader observe a partially-populated entry. Population now completes into
    a local map first, then publishes atomically.

Testing

  • New AstPropertyResolveUtilsSpec.groovy (public-API only): includes a test that
    constructs two distinct ClassNodes sharing the exact same name and asserts each
    resolves independently — a direct, deterministic reproduction of the collision
    mechanism, not a rerun-and-hope test.
  • :grails-datamapping-core:test (full module): BUILD SUCCESSFUL, no failures,
    including both previously-flaky feature methods.
  • CodeNarc/Checkstyle: clean.

Related: #16030

…ache by ClassNode identity

WhereQueryClosureCaptureSpec ("generated closure constructors are identical
across compilations" and "association criteria closures capture only the
variables they reference") was ~2% flaky in CI with no hard failures on the
same commit - a classic sign of shared, racy static state rather than a test
bug.

Root cause: AstPropertyResolveUtils.cachedClassProperties is a static,
process-wide java.util.HashMap that the where{} query transform
(DetachedCriteriaTransformer) consults to resolve a domain class's property
names/types. Two problems compounded:

1. Not thread-safe. Gradle runs many spec classes concurrently within one
   JVM/fork, so concurrent put()/resize on a plain HashMap can corrupt its
   internal structure - a well known source of nondeterministic, rare
   failures that reproduce inconsistently between reruns.

2. Keyed by name (ClassNode#getName()), and ClassNode#equals()/hashCode()
   themselves compare by name too. Two distinct ClassNode instances that
   share a name - e.g. the same source compiled twice into separate
   GroovyClassLoaders, as WhereQueryClosureCaptureSpec's "identical across
   compilations" test does on purpose, or any two test fixtures compiled
   without a package - collide on the same cache entry. Whichever
   compilation populates the entry first "wins", so the second compilation's
   property resolution (and therefore the generated closure's captured
   variables) can silently depend on stale data from an unrelated
   ClassNode/classloader.

Fix: back the cache with an IdentityHashMap (keys compared by `==`, not
`equals()`), wrapped in Collections.synchronizedMap for thread safety. This
eliminates both the corruption hazard and the name-collision hazard
outright, since two different ClassNode instances can never share an entry
regardless of what they're named. Also stopped publishing the new entry into
the shared map until it is fully populated, so a concurrent reader can never
observe a partially-built entry.

Note: the cache already keyed by ClassNode#getName() (the fully-qualified
name), not a bare simple name as initially suspected - but for classes
compiled without a package (common in tests and generated sources), the
qualified name *is* the simple name, so name-based keying could not have
fixed the collision this spec's own "compile the same source twice" test
depends on. Identity-based keying removes the ambiguity entirely instead of
narrowing it.

Added AstPropertyResolveUtilsSpec, a new unit test that builds two distinct
ClassNode instances with the same unqualified name and different property
sets, and proves each resolves and caches its own properties independent of
the other - directly reproducing and proving the fix for the collision,
without relying on reruns of the flaky spec to catch it.

Verified: :grails-datamapping-core:test passes in full, including
WhereQueryClosureCaptureSpec and the new AstPropertyResolveUtilsSpec.
codeStyle (Checkstyle + CodeNarc) reports no violations.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 21:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses flakiness in WhereQueryClosureCaptureSpec by fixing AstPropertyResolveUtils’s static property-cache to avoid key collisions between distinct ClassNode instances that compare equal by name, and by ensuring cache entries are only published once fully populated.

Changes:

  • Switch cachedClassProperties to a synchronized IdentityHashMap keyed by ClassNode identity (==) instead of name/equals().
  • Avoid publishing partially-populated cache entries by building into a local map first, then publishing to the shared cache.
  • Add a new Spock spec to deterministically reproduce and guard against same-name ClassNode cache collisions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
grails-datamapping-core/src/main/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtils.java Reworks the static cache to use identity-based keys and fixes non-atomic publication of newly created cache entries.
grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtilsSpec.groovy Adds tests that exercise same-name ClassNode scenarios to prevent future cache-collision regressions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bito-code-review

Copy link
Copy Markdown

The test case "property lookups for two same-named ClassNodes in different packages do not corrupt each other" currently uses different fully-qualified names ('org.example.one.Widget' and 'org.example.two.Widget'). To better exercise the regression mechanism and ensure the fix works for same-named classes, you should update this test to use the same fully-qualified name for both ClassNode instances. This will confirm that the IdentityHashMap correctly distinguishes between distinct instances that share the same name.

grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtilsSpec.groovy

given: 'two distinct ClassNodes with the same simple name declared in different packages'
        ClassNode first = new ClassNode('org.example.Widget', Modifier.PUBLIC, ClassHelper.OBJECT_TYPE)
        first.addProperty('color', Modifier.PUBLIC, ClassHelper.STRING_TYPE, null, null, null)

        ClassNode second = new ClassNode('org.example.Widget', Modifier.PUBLIC, ClassHelper.OBJECT_TYPE)
        second.addProperty('weight', Modifier.PUBLIC, ClassHelper.Integer_TYPE, null, null, null)

@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 51.8557%. Comparing base (6d1acad) to head (c3a4d92).
⚠️ Report is 477 commits behind head on 8.0.x.

Files with missing lines Patch % Lines
...astore/gorm/transform/AstPropertyResolveUtils.java 92.8571% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@               Coverage Diff                @@
##             8.0.x     #16034         +/-   ##
================================================
+ Coverage         0   51.8557%   +51.8557%     
- Complexity       0      18112      +18112     
================================================
  Files            0       2046       +2046     
  Lines            0      96273      +96273     
  Branches         0      16726      +16726     
================================================
+ Hits             0      49923      +49923     
- Misses           0      38981      +38981     
- Partials         0       7369       +7369     
Files with missing lines Coverage Δ
...astore/gorm/transform/AstPropertyResolveUtils.java 80.7229% <92.8571%> (ø)

... and 2045 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@borinquenkid

Copy link
Copy Markdown
Member Author

Thanks for the review! The scenario you're describing — two distinct ClassNode instances sharing the exact same fully-qualified name — is already covered by the test right below this one:

void "property lookups for two distinct ClassNode instances with the exact same unqualified name do not corrupt each other"() {
    given: 'two distinct ClassNode instances - as produced by two separate compilations - sharing an identical unqualified name'
    ClassNode first = new ClassNode('Widget', Modifier.PUBLIC, ClassHelper.OBJECT_TYPE)
    first.addProperty('color', Modifier.PUBLIC, ClassHelper.STRING_TYPE, null, null, null)

    ClassNode second = new ClassNode('Widget', Modifier.PUBLIC, ClassHelper.OBJECT_TYPE)
    second.addProperty('weight', Modifier.PUBLIC, ClassHelper.Integer_TYPE, null, null, null)

    expect: 'the two ClassNode instances compare equal by name - the exact condition that would collide in a name-keyed or equals()-keyed cache'
    first == second
    first.hashCode() == second.hashCode()
    !first.is(second)
    ...

It goes a step further than the suggested edit by asserting first == second and matching hashCode(), which explicitly proves the collision condition a name-/equals()-keyed cache would hit — exactly the regression this fix guards against.

The test this comment is attached to intentionally covers a different case: two ClassNodes that share a simple name but differ by package (so they're not equal), which is a distinct scenario worth keeping separate. Given the exact-FQN-collision case is already exercised, I'll leave both tests as-is rather than introduce a near-duplicate.

@jamesfredley jamesfredley moved this to Todo in Apache Grails Jul 24, 2026
@borinquenkid borinquenkid added this to the grails:8.0.0-RC1 milestone Jul 25, 2026
@borinquenkid borinquenkid removed the status in Apache Grails Jul 25, 2026
@borinquenkid borinquenkid moved this to Todo in Apache Grails Jul 25, 2026

@jdaugherty jdaugherty left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had AI review this before taking a look, here's it's comments:

Reviewed the identity-keyed cache change. The direction is right — ClassNode.equals/hashCode really do compare by getText() (confirmed in Groovy 5.0.7: equals falls through to ((ClassNode) that).getText().equals(getText())), so a name- or equals()-keyed static cache genuinely conflates distinct compilations, and reordering so an entry is published only once fully populated is a real fix.

Two things I'd want resolved before this lands:

  1. Keying a static map by ClassNode turns a bounded cache into a classloader leak. A primary ClassNode transitively pins its CompileUnit and therefore its GroovyClassLoader, and nothing ever evicts. Detail inline — the per-node getNodeMetaData alternative removes the collision and the leak, and drops the static state entirely. This is the part I'm most concerned about, since the whole point of the change is to make a process-wide static cache safe.
  2. The stated root cause doesn't explain a flaky failure. The same source compiled twice yields two ClassNodes with identical property sets, so a name collision between just those two can't diverge. The genuinely order-dependent input is classNode.isResolved() at first-lookup time, which this PR doesn't change. Inline.

Smaller items, all inline: the protected field's type change plus new final is a breaking change worth converting into private + an explicit clear hook; the javadoc's concurrency rationale cites parallel test forks, which are separate JVMs and so cannot race on a static field; and the new spec doesn't cover the domain-class branches or the concurrency behaviour it claims to fix.

One follow-up outside the diff: WhereQueryClosureCaptureSpec:35 and WhereQueryEmbeddedBlockTransformSpec:37 both still carry the comment "The domain class names must be unique across the test JVM because AstPropertyResolveUtils caches resolved properties statically by class name". That is no longer true after this change, and the fixture renaming in 08bd0bf that it justifies is now unnecessary. Please update both comments (and note whether the renaming can be reverted) so those specs aren't left documenting the old behaviour.

Comment on lines +77 to +78
first == second
first.hashCode() == second.hashCode()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two assert Groovy's own ClassNode equality contract rather than anything about AstPropertyResolveUtils. They hold today — 5.0.7's ClassNode.equals compares getText() and hashCode() delegates to getText().hashCode() — but if Groovy ever moves ClassNode to identity equality this spec fails while the production behaviour it guards is still perfectly correct.

!first.is(second) on line 79 is the precondition the test actually needs. Consider keeping that and demoting the other two to a comment explaining why two same-named nodes used to collide.

@borinquenkid borinquenkid Jul 31, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - trimmed to just !first.is(second), with a comment explaining why the equals()/hashCode() equality matters (it's the exact collision condition a name- or equals()-keyed cache would hit) rather than asserting on Groovy's own ClassNode equality contract.

borinquenkid and others added 2 commits July 30, 2026 21:03
…st gaps

Responds to review feedback on the AstPropertyResolveUtils identity-cache fix:

- Cache resolved properties as ClassNode metadata (classNode.redirect()
  .getNodeMetaData(key, fn)) instead of a static IdentityHashMap. This removes
  the classloader leak a static, never-evicted map would cause in long-lived
  JVMs (Gradle daemon, dev-mode recompiles) - cached data now becomes eligible
  for GC together with the ClassNode/compilation it describes. It also removes
  the now-unnecessary protected static field entirely (no more breaking-change
  surface on it), and eliminates the identity-collision hazard by construction
  rather than by choice of key type, since each ClassNode owns its own cache
  slot with no shared keyspace to collide on.

- Corrected the javadoc's concurrency rationale: parallel Gradle test forks are
  separate JVMs (maxParallelForks) and this build does not enable JUnit's
  in-JVM parallel execution, so they were never a real race. Documented the
  actual, narrower exposure and why per-ClassNode storage sidesteps it (a
  given ClassNode is only ever populated by the single thread compiling it).

- Verified classNode.isResolved() cannot flip from false to true for a given
  ClassNode instance post-construction (ClassNode.clazz has no setter outside
  the ClassNode(Class) constructor - checked against Groovy 5.0.7 sources), so
  the resolve-once-cache-forever design does not have the stale-snapshot
  hazard it would otherwise risk for classes resolved at varying compile
  phases.

- Updated the stale "must be unique" comments in WhereQueryClosureCaptureSpec
  and WhereQueryEmbeddedBlockTransformSpec: per-instance caching means no
  same-named fixture can collide regardless of naming; the distinctive names
  are kept for readability, not correctness.

- Extended AstPropertyResolveUtilsSpec: domain-class identity/version
  injection and hasMany/belongsTo/hasOne resolution via both AST initial
  expressions and (for an already-compiled class) reflection; a concurrency
  test resolving many distinct, identically-named ClassNodes across threads
  simultaneously; a test proving getPropertyNames returns the cached snapshot
  rather than recomputing after the ClassNode is mutated post-cache. Trimmed
  the equals()/hashCode() assertions in the collision test to the actual
  precondition the cache depends on (!first.is(second)), with a comment
  explaining why the equals()/hashCode() equality is the interesting/dangerous
  part rather than something to assert on.

Verified: :grails-datamapping-core:test passes in full (including 3 repeated
runs of AstPropertyResolveUtilsSpec to rule out flakiness in the new
concurrency test). codeStyle (Checkstyle + CodeNarc) reports no violations
across the whole repo.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Before pushing, ran an adversarial review of the prior commit's per-ClassNode
cache redesign. It found the new concurrency test didn't test concurrency at
all (all threads operated on distinct ClassNode instances, so nothing was
actually shared), and, digging into why that mattered, surfaced a genuine gap:
getNodeMetaData(key, fn) is backed by ListHashMap, which is explicitly
documented as not thread-safe, and some real callers (any property typed
Object/def) can resolve to ClassHelper.OBJECT_TYPE/STRING_TYPE - small,
JVM-wide-shared singleton nodes touched by every compilation in the process,
not scoped to one thread the way the previous javadoc assumed.

- Synchronize per-ClassNode (synchronized (cacheHolder) in
  getPropertiesFromCache) so concurrent calls into this class are safe with
  respect to each other, including on a shared singleton node. Documented
  precisely what this does and does not cover (it can't force unrelated
  compiler code writing a different metadata key to the same node to
  synchronize on the same monitor - that residual risk belongs to ClassNode's
  metadata storage in general).

- Replaced the placebo concurrency test (distinct nodes only) with one that
  actually shares a single ClassNode across 32 threads, plus kept the
  distinct-node test since it still legitimately covers the identity-collision
  property. Verified the shared-node test's value honestly: with the
  synchronized guard removed, it still passed 8/8 runs, because the cached
  computation is deterministic/idempotent, so a black-box return-value test
  can't reliably force ListHashMap's undocumented internals into an observably
  wrong state. Said so directly in the test's comment rather than overclaiming
  what it proves - the synchronization is justified by ListHashMap's own
  "not thread-safe" documentation, not by this test catching a live bug.

- Tightened the isResolved()-invariance javadoc: the previous version's
  argument ("clazz has no setter, so isResolved() can't flip") was incomplete,
  since isResolved() also delegates through redirect(), which can in principle
  be reassigned after construction. Verified via an actual GroovyBugError at
  test runtime that ClassNode.setRedirect() refuses to run on a primary node -
  and every ClassNode this utility's real callers pass in during an AST
  transform is primary - so redirect() is simply the node itself for the
  entire object's life in every real usage. Tried to write a unit test proving
  the redirect/self-healing behavior for the one remaining theoretical case
  (a non-primary reference node) and confirmed such a node can't be
  constructed from outside Groovy's own ast package without reflection, which
  would violate this repo's public-API-only testing rule - dropped that test
  and rely on the (now precise) javadoc instead.

Verified: :grails-datamapping-core:test passes in full from a clean checkout
of the correct worktree/branch (caught and corrected a shell cwd mixup that
had briefly pointed a test run at a different local branch entirely).
codeStyle (Checkstyle + CodeNarc) reports no violations across the whole repo.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@borinquenkid
borinquenkid requested a review from jdaugherty July 31, 2026 18:00
@testlens-app

testlens-app Bot commented Jul 31, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: c3a4d92
▶️ Tests: 57639 executed
⚪️ Checks: 62/62 completed


Learn more about TestLens at testlens.app.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (4)

grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtilsSpec.groovy:270

  • As with the previous test, a timeout/exception during Future#get(...) will skip executor.shutdown(), leaking worker threads into the rest of the test run. Wrapping the concurrent section in try/finally ensures the ExecutorService is always terminated.
        List<List<String>> results = futures.collect { Future<List<String>> future -> future.get(30, TimeUnit.SECONDS) }
        executor.shutdown()

grails-datamapping-core/src/main/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtils.java:56

  • The PR description says the fix switches the global cache to a synchronized IdentityHashMap, but the implementation here no longer uses a static map at all; it caches via ClassNode node-metadata (PROPERTIES_CACHE_KEY). Please update the PR description (or the code) so reviewers and future archeology don’t get conflicting information about what was actually changed.
     * Key under which the resolved property map is stashed via {@link ClassNode#getNodeMetaData(Object, java.util.function.Function)}.
     * <p>
     * Earlier versions of this class cached resolved properties in a single static, process-wide
     * {@code Map} keyed by class name (later by {@code ClassNode} identity). Both designs share a
     * problem: a static map is never emptied, so every {@code ClassNode} ever looked up - and, for

grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtilsSpec.groovy:205

  • This spec creates a GroovyClassLoader but never closes it, which can retain loaded classes and contribute to memory pressure across the test suite. Also, if the class lookup fails, ClassHelper.make(domainClass) will NPE with a less-informative error. Consider asserting the class was found and closing the loader in a cleanup block.
        Class<?> domainClass = gcl.loadedClasses.find { it.simpleName == 'ReflectedAssociationFixture' }

        and: 'a fresh ClassNode built from the already-compiled class, as happens once compilation has finished'
        ClassNode resolvedNode = ClassHelper.make(domainClass)

grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtilsSpec.groovy:227

  • If any Future#get(...) throws (timeout/interruption), the test will fail before executor.shutdown() runs, leaking the thread pool into the rest of the JVM and potentially impacting later tests. Put executor shutdown into a finally block (and consider shutdownNow + awaitTermination) so threads are always cleaned up.

This issue also appears on line 268 of the same file.

        List<Boolean> outcomes = futures.collect { Future<Boolean> future -> future.get(30, TimeUnit.SECONDS) }
        executor.shutdown()

@borinquenkid
borinquenkid requested a lite review from Copilot August 16, 2026 15:43
@borinquenkid

Copy link
Copy Markdown
Member Author

@jdaugherty

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (3)

grails-datamapping-core/src/main/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtils.java:60

  • The PR description says the fix switches the cache to a synchronized IdentityHashMap keyed by ClassNode identity, but the implementation here has instead moved the cache to per-ClassNode node metadata (PROPERTIES_CACHE_KEY via getNodeMetaData). Please update the PR description (and/or title) to reflect the actual approach so reviewers and future archaeology don’t get misled.
     * Earlier versions of this class cached resolved properties in a single static, process-wide
     * {@code Map} keyed by class name (later by {@code ClassNode} identity). Both designs share a
     * problem: a static map is never emptied, so every {@code ClassNode} ever looked up - and, for
     * a primary node, the {@code GroovyClassLoader}/{@code CompileUnit} it pins via
     * {@link ClassNode#getModule()} - is retained for the lifetime of the JVM. In a long-lived
     * process that repeatedly compiles Groovy (a Gradle daemon reusing its Groovy compiler across
     * builds, a dev-mode recompile loop), that is an unbounded classloader leak.

grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtilsSpec.groovy:270

  • Same issue as above: if a Future.get(...) times out or barrier.await() throws, executor.shutdown() is skipped and the fixed thread pool can keep the JVM alive. Use try/finally to always shut down the pool (prefer shutdownNow + awaitTermination in tests).
        int threadCount = 32
        ExecutorService executor = Executors.newFixedThreadPool(threadCount)
        CyclicBarrier barrier = new CyclicBarrier(threadCount)

        when: 'all threads race to resolve properties for the same instance at once'
        List<Future<List<String>>> futures = (0..<threadCount).collect {
            executor.submit({ ->
                barrier.await()
                AstPropertyResolveUtils.getPropertyNames(sharedNode)
            } as Callable<List<String>>)
        }
        List<List<String>> results = futures.collect { Future<List<String>> future -> future.get(30, TimeUnit.SECONDS) }
        executor.shutdown()

grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/transform/AstPropertyResolveUtilsSpec.groovy:227

  • If any Future.get(...) or barrier.await() throws, the test exits before executor.shutdown() runs, leaving non-daemon pool threads alive and potentially hanging the test JVM. Wrap the concurrent section in try/finally and always shut down/await termination.
        int threadCount = 20
        ExecutorService executor = Executors.newFixedThreadPool(threadCount)
        CyclicBarrier barrier = new CyclicBarrier(threadCount)

        when: 'all threads race to populate the cache for their own instance at the same time'
        List<Future<Boolean>> futures = (0..<threadCount).collect { int i ->
            executor.submit({ ->
                barrier.await()
                ClassNode node = new ClassNode('ConcurrentWidget', Modifier.PUBLIC, ClassHelper.OBJECT_TYPE)
                String propertyName = "prop${i}".toString()
                node.addProperty(propertyName, Modifier.PUBLIC, ClassHelper.STRING_TYPE, null, null, null)

                List<String> names = AstPropertyResolveUtils.getPropertyNames(node)
                names.contains(propertyName) && names.count { it.startsWith('prop') } == 1
            } as Callable<Boolean>)
        }
        List<Boolean> outcomes = futures.collect { Future<Boolean> future -> future.get(30, TimeUnit.SECONDS) }
        executor.shutdown()

@jdaugherty jdaugherty left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after the move to per-ClassNode metadata. Everything from the previous round is addressed and I have resolved those threads — the static map is gone, publication is safe, and the tests now reach the domain-class branches. The empirical reproduction you did settled the question of whether the flake mechanism was real, which was the right way to answer it.

Two major remain:

  1. AstUtils.isDomainClass is @Memoized by name, so "no collision is possible" and "no leak is possible" are not accurate yet. I verified a genuinely @Entity node losing its id/version because a same-named plain node was resolved first. The fix is still a clear improvement — it narrows the corruption from the whole property map to a single boolean — but the javadoc and the two spec comments should say what is actually true. The retracted "names must be unique" warning is the one I most want restored, since Book/Author really are shared across specs in this module and forkEvery = 50 puts them in one JVM.

  2. !first.is(second) in the new spec sits in a given:/and: block, so it never asserts.

The rest is smaller: the cache now writes into interned ClassHelper singletons that ClassNode.getModule() reads without the lock, alien code runs under the node monitor, and the new spec leaks its executors and classloader on the failure path.


private static Map<String, ClassNode> computeProperties(ClassNode classNode) {
Map<String, ClassNode> newProperties = new HashMap<>();
boolean isDomainClass = AstUtils.isDomainClass(classNode);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AstUtils.isDomainClass(ClassNode) is @Memoized (AstUtils.groovy:428), and that memo cache is keyed by ClassNode equality, not identity — ClassNode.equals/hashCode both reduce to getText() (ClassNode.java:1283/1288 in 5.0.7). @Memoized also defaults to maxCacheSize = 0, so it is an unbounded UnlimitedConcurrentCache that never evicts.

That leaves both headline claims in the new javadoc false one level down: two distinct same-named ClassNodes do still collide here, and the first node seen per distinct name is retained for the life of the JVM, pinning its module, compile unit and GroovyClassLoader with it.

Verified against this branch with two hand-built nodes both named probe.MemoWidget, the first plain and the second genuinely carrying @Entity, resolved in that order:

plainProps            = [title]
entityProps           = [title]     <- @Entity node, no id/version
isDomainClass(entity) = false       <- despite the annotation

Control, the same @Entity node under a name nothing else had touched: [id, title, version].

To be fair to the change: the @Memoized predates this PR and you are not introducing it. But this PR does newly route through it. Under the old name-keyed map, a same-named second node returned the first node's cached map and never called isDomainClass at all; now it computes its own properties and consults the memoized verdict. The corruption narrows from the entire property map to the domain-ness boolean, which is a genuine improvement — "no collision is possible" just overstates where it lands.

I am not asking you to fix isDomainClass here; that deserves its own issue rather than growing this PR. Just asking that the javadoc claims match what is actually true.

}
} return cachedProperties;
ClassNode cacheHolder = classNode.redirect();
synchronized (cacheHolder) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synchronized (cacheHolder) cannot deliver the safety the javadoc claims for it, because the nodes being locked are not always private to one compilation. QueryStringTransformer.groovy:208 walks a property path by feeding each resolved type back in as the next receiver, so a JDK-typed property makes an interned ClassHelper singleton the cache holder.

Verified on this branch:

propType.is(ClassHelper.STRING_TYPE) = true
STRING_TYPE cache metadata before    = false
STRING_TYPE cache metadata after     = true

ClassNode.getModule() reads that same ListHashMap through getNodeMetaData(ModuleNode.class) without taking the node monitor, so this lock only excludes callers that opt into it. The javadoc frames the leftover exposure as risk that "belongs to ClassNode's metadata storage in general", but before this change the utility never wrote to any ClassNode — it is this PR that makes GORM a writer to those JVM-wide maps.

Gating on cacheHolder.isPrimaryClassNode() and skipping the cache otherwise would confine writes to nodes the compilation actually owns, and the JDK types that get skipped are cheap to recompute.

Separately, computeProperties runs entirely inside this monitor, and for a resolved domain class it reaches ClassPropertyFetcher.forClass(getTypeClass()).getPropertyValue(...), which forces the user class's static initialiser and invokes user-written static getters while the lock is held. Computing outside the lock and publishing afterwards would take alien code out from under it.

Map<String, ClassNode> newProperties = new HashMap<>();
boolean isDomainClass = AstUtils.isDomainClass(classNode);
if (isDomainClass) {
newProperties.put(GormProperties.IDENTITY, new ClassNode(Long.class));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: both of these are redirect-less holder nodes, so under the new scheme each becomes its own cache holder and never shares with anything. Line 236 already uses the plain-reference convention — ClassHelper.make(Long.class).getPlainNodeReference() here would redirect to the interned Long node so id/version lookups share a single entry again.

// because AstPropertyResolveUtils used to cache resolved properties in a single
// static map keyed by class name, so a same-named fixture in another spec could
// collide with this one. AstPropertyResolveUtils now caches per-ClassNode instance
// (see its javadoc), so that collision can no longer happen regardless of naming -

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of everything here, this is the change I would most like reverted. Given the @Memoized isDomainClass collision (see the comment on computeProperties), uniqueness is still required for correctness here — the original warning was right.

The risk is concrete inside this module. DirtyCheckTransformationSpec compiles @DirtyCheck class Book and class Author from strings, which are not domain classes, while QueryStringTransformerSpec compiles @Entity class Book and class Author, which are. forkEvery = 50 on CI (gradle/test-config.gradle:88) means up to fifty spec classes share one JVM, so both can land in the same fork in either order, and whichever runs first wins the memoized isDomainClass answer for that name.

A maintainer who takes this comment at face value and renames ClosureCaptureBook back to Book walks straight into an order-dependent flake of exactly the #16030 character.

Please keep the "must be unique" wording as a requirement. If you want to record the history, the accurate version is that the property map is now per-instance while the domain-class verdict is still name-keyed.

// because AstPropertyResolveUtils used to cache resolved properties in a single
// static map keyed by class name, so a same-named fixture in another spec could
// collide with this one. AstPropertyResolveUtils now caches per-ClassNode instance
// (see its javadoc), so that collision can no longer happen regardless of naming -

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same point as the equivalent comment in WhereQueryClosureCaptureSpec: "that collision can no longer happen regardless of naming" does not hold while AstUtils.isDomainClass is @Memoized by name, so the uniqueness requirement should stay stated as a requirement rather than a stylistic preference.

// keyed by name or by equals()/hashCode() would treat them as the same entry; only
// reference identity (!first.is(second)) tells them apart, which is exactly what the
// cache must key on.
!first.is(second)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This precondition never runs as an assertion. It sits in an and: block continuing given:, and Spock applies implicit conditions only in then:/expect: blocks, so the expression is evaluated and discarded.

Confirmed on this branch by putting false and x == 999 in an and: after given: in a scratch spec — the feature passed.

Needs assert !first.is(second), or to move into an expect: block. Worth correcting because the comment directly above presents this line as the load-bearing premise of the feature.

} as Callable<Boolean>)
}
List<Boolean> outcomes = futures.collect { Future<Boolean> future -> future.get(30, TimeUnit.SECONDS) }
executor.shutdown()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

executor.shutdown() is the last statement of the when: block, so it is skipped on exactly the path where it matters: if any future.get(30, SECONDS) throws, the collect aborts and the pool is never shut down. Executors.newFixedThreadPool threads are non-daemon and do not time out, and with forkEvery = 50 this JVM goes on to run more specs.

barrier.await() also has no timeout, so if one worker dies before reaching the barrier the rest park indefinitely and the only symptom is a 30-second timeout per future.

Suggest a cleanup: block calling executor.shutdownNow() in both concurrency features, plus the timeout overload of await. The GroovyClassLoader in the reflection feature is also never closed — it is AutoCloseable, and closing it would suit a spec whose subject is a classloader leak.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

4 participants