test(ofrep): run the provider conformance suite against the OFREP provider - #1612
Draft
aepfli wants to merge 1 commit into
Draft
test(ofrep): run the provider conformance suite against the OFREP provider#1612aepfli wants to merge 1 commit into
aepfli wants to merge 1 commit into
Conversation
…vider
The OFREP provider is stateless, and the capability declaration says so: Object
and nothing else. Its whole surface is a constructor, onClose and four
resolve*Evaluation methods that each POST to /ofrep/v1/evaluate/flags/{key} --
ofrep-provider.ts:15-111. There is no initialize, no events emitter and no cached
ruleset, so Events, Stale, ConfigurationChange and UnavailableInit are all
undeclared for the same root cause rather than four separate ones, and every
omission is commented against the line that establishes it.
That is narrower than the Go and Java OFREP providers, which declare Object and
StrictNumericTyping. The extra one is withheld here for a language reason rather
than a provider one: JavaScript has no integer type, so asking for float-flag as
an Integer is indistinguishable from asking for it as a Float and the scenario is
unsatisfiable by construction. It must stay undeclared for every JavaScript
provider -- see the TCK README.
23 of the 29 scenarios run and pass; the other 6 report as skipped with their
reason in the test name. Go and Java run 24, the difference being exactly the
numeric one.
No new test infrastructure. flagd serves OFREP on port 8016 and the testbed image
already exposes it, so the suite drives the same launchpad control API the flagd
adoption uses. FlagdComposeContainer was not reusable -- it lives inside another
Nx project's src/ with no export and types its port accessor in terms of flagd's
ResolverType -- so this uses testcontainers directly, and skips the Compose file's
Envoy proxy, which serves flagd's gRPC cases and is pure startup cost for an HTTP
protocol. Only the testbed version is shared, read from the submodule so both
suites move together.
libs/providers/ofrep-web is deliberately not adopted. It is the only OFREP
provider here with events, STALE and a failable init, but flagd's OFREP handler
never writes an ETag, never reads If-None-Match, and its bulk response carries no
eventStreams field, so its polling and SSE paths are both unreachable against this
backend. Adopting it here would declare capabilities the backend, not the
provider, makes untestable. It waits for a neutral OFREP testbed.
Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs the provider conformance suite against the OFREP provider, pointed at flagd's OFREP API.
Stacked on #1608, which adopts the suite for flagd. Part of open-feature/spec#417.
What it declares, and what that says about OFREP
One capability out of nine, and every omission is a fact about the provider rather than a convenience.
No
@events, and therefore no@lifecycleor@stale. OFREP is a stateless HTTP protocol: the provider resolves every flag over the wire and has no connection to lose, no stream to watch, and nothing to announce.That is exactly the case that motivated splitting
@lifecycleout of@events. Before the split, the readiness scenario ran for anything declaring@events— and the SDK synthesisesPROVIDER_READYfor a provider that does not implement state handling, on the reasoning that such a provider can be assumed ready immediately. A stateless provider would therefore have passed the readiness scenario without demonstrating anything; a no-op provider passes it identically. Here it declares neither tag and those scenarios are reported as skipped with the reason.No
@configuration-change— there is nothing to notice a change with. A polling OFREP client could plausibly declare it; this one does not poll.No
@unavailable— the provider performs no initialisation that reaches the backend, so there is no initialisation to fail.No
@strict-numeric-typing, and this is the one difference from the other three languages. JavaScript has no integer type, so no JavaScript provider can distinguish an integer from a float. Go, Java and Python all declare it for their OFREP providers. This is a property of the language, not a gap in the provider — which is precisely why the conformance report has anot-applicableoutcome distinct fromnot-declared: reporting it as "not declared" would show every JavaScript provider as missing something none of them can ever have.The backend
Driven through the same HTTP control API as the flagd suite, since flagd serves OFREP alongside its other resolvers. Pointing two different providers at one backend means a difference in results is attributable to the provider rather than to the backend, which is what makes the two adoptions comparable.
Cross-language convergence
@object,@strict-numeric-typing@object,@strict-numeric-typing,@lifecycle@object,@strict-numeric-typing@objectFour teams reached the same conclusion about one protocol from four runtimes, independently. The JavaScript delta is entirely the language. That convergence is what makes a future disagreement meaningful.