Fix OpenFeature provider hook test isolation

[leoromanovsky]

Motivation

Two ProviderTest cases produce false failures on master when the experimental span-enrichment gate is enabled in the test environment. The provider correctly registers an additional optional hook in that configuration, but the tests asserted that the complete hook list always contained exactly one entry. These CI Visibility failures create noise for contributors and can hide real OpenFeature regressions.

Changes

The provider-hook test now verifies the invariant it actually owns: the flag-evaluation metrics hook is registered whether span enrichment is disabled or enabled. It exercises both gate states explicitly and checks hook membership instead of relying on list size or ordering.

The shutdown test now has a name and assertion aligned with its behavior. It verifies evaluator shutdown and no longer couples lifecycle coverage to the number of optional provider hooks.

Decisions

This is intentionally a test-only change because production behavior is correct: enabling span enrichment should add a hook. The tests reuse the existing explicit gate seam to cover both configurations deterministically, while the dedicated span-enrichment tests remain responsible for the optional hook's own registration behavior. No global environment mutation or timeout adjustment is introduced.

Validation

• ./gradlew :products:feature-flagging:feature-flagging-api:test --tests datadog.trace.api.openfeature.ProviderTest :products:feature-flagging:feature-flagging-api:spotlessCheck -Prerun.tests.feature-flagging-api -PtestJvm=11 --no-daemon --stacktrace
• DD_EXPERIMENTAL_FLAGGING_PROVIDER_SPAN_ENRICHMENT_ENABLED=true ./gradlew :products:feature-flagging:feature-flagging-api:test --tests datadog.trace.api.openfeature.ProviderTest -Prerun.tests.feature-flagging-api -PtestJvm=11 --no-daemon --stacktrace

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.98 s	14.07 s	[-1.4%; +0.1%] (no difference)
startup:insecure-bank:tracing:Agent	12.93 s	13.03 s	[-1.5%; +0.0%] (no difference)
startup:petclinic:appsec:Agent	17.61 s	17.09 s	[+1.9%; +4.2%] (significantly worse)
startup:petclinic:iast:Agent	17.53 s	17.70 s	[-1.6%; -0.2%] (maybe better)
startup:petclinic:profiling:Agent	17.58 s	17.33 s	[+0.1%; +2.8%] (maybe worse)
startup:petclinic:sca:Agent	17.57 s	17.55 s	[-0.9%; +1.1%] (no difference)
startup:petclinic:tracing:Agent	16.29 s	16.74 s	[-7.1%; +1.7%] (no difference)

Commit: 56cd3f1e · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[datadog-datadog-us1-prod]

More details

The revised tests remain deterministic when span enrichment is unset, enabled, or disabled. Exact hook-list validation confirmed the metrics hook is always present, enrichment appears only when explicitly enabled, and evaluator shutdown no longer depends on optional-hook count.

Was this helpful? React 👍 or 👎

_{📊 Validated against 10 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 56cd3f1 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[dd-octo-sts]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-08-10 19:57:19 UTC ℹ️ Start processing command /merge

---

2026-08-10 19:57:24 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

---

2026-08-10 20:41:45 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 79acdfa:

• test_inst_latest: [21, 1/6]

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

• Investigate the failures and when ready, re-add your pull request to the queue!
• If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
• Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

[leoromanovsky]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-08-11 01:54:21 UTC ℹ️ Start processing command /merge

---

2026-08-11 01:54:26 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

---

2026-08-11 02:31:37 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 48dd94d:

• test_inst: [25, 7/8]

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

• Investigate the failures and when ready, re-add your pull request to the queue!
• If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
• Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.