Skip to content

test(ofrep): adopt the OpenFeature Provider TCK - #1840

Draft
aepfli wants to merge 17 commits into
feat/provider-tck-flagdfrom
feat/provider-tck-ofrep
Draft

test(ofrep): adopt the OpenFeature Provider TCK#1840
aepfli wants to merge 17 commits into
feat/provider-tck-flagdfrom
feat/provider-tck-ofrep

Conversation

@aepfli

@aepfli aepfli commented Aug 24, 2026

Copy link
Copy Markdown
Member

Stacked on #1830. Base is feat/provider-tck, so this diff shows only the new work. Review
#1830 first — it describes the suite, the adoption API and the capability vocabulary, and none of
that is repeated here.

Part of #1829 and open-feature/spec#417.

The OFREP provider under the conformance suite. Three files: one test class, a Compose file, and the
test dependency in providers/ofrep/pom.xml.

No new infrastructure. The same flagd-testbed image the flagd suites use, exposing 8016 (OFREP)
and 8080 (launchpad), no pinned host ports.

providers/ofrep's e2e package is gone — the conformance suite had been its only member. That
is the nesting argument in its clearest form: the adoption was filed under a category the module did
not have. What remains is …/ofrep/tck/OfrepTest.java, selected by **/tck/*.java in both the default
build's exclusions and the tck profile's includes.

What it declares

Declared: @object, @variants, @targeting, @standard-reasons.

Withheld: @lifecycle, @reinitialization, @events, @stale, @configuration-change,
@unavailable, @disabled-flags, @numeric-coercion. (@large-integers is refused by the
implementation for every Java provider — a 32-bit Integer accessor — rather than withheld here.)

Six of those share one root cause: OfrepProvider is a bare FeatureProvider with no lifecycle of
its own. It holds no state, opens no stream, runs no poll loop and does not override initialize()
every evaluation is a fresh, independent HTTP POST. There is nothing in it that could observe a
backend transition, let alone report one. Declaring @lifecycle would make the readiness scenario
pass vacuously: the PROVIDER_READY a client sees is the SDK's, emitted identically for
NoOpProvider and for a provider aimed at a dead port.

Two are more interesting, and both were measured rather than read off the source:

  • @numeric-coercion is withheld and carries no deviation. Its three scenarios test all three
    cases and a provider declaring the tag must pass all three; this one passes one, and gets it right
    for the wrong reason — 0.5 as an integer is rejected because it is a Double, not because it is
    fractional. Both lossless cases fail on the same exact-instance check. No deviation accompanies it
    deliberately: Appendix F is explicit that this is the one capability the specification does not
    define — OpenFeature has a single number type, the rule is borrowed from flagd's ADR, and "a
    provider that behaves differently is not violating the specification"
    .
  • @disabled-flags is withheld and does carry a deviation, which is the opposite call, and the
    difference is where the rule lives. flagd's OFREP endpoint answers a disabled flag with 200 and
    {"key":…,"reason":"DISABLED"} and no value member. That is well-formed OFREP — the
    codeDefaultFlag member schema, described as "A flag evaluation that defers to the code default
    value … The provider must use the code default value when processing this response"
    — and
    handleResolved reaches its responseValue == null branch and reports FLAG_NOT_FOUND, discarding
    the reason it parsed one field earlier. So a MUST in the protocol this provider implements is
    unmet, and the gap is wider than the four rows that found it: every codeDefaultFlag response is
    reported to the application as a failure on an evaluation that succeeded.

That last one is the declaration on this branch the settled guidance would shape differently, and it
is recorded rather than quietly left.
KnownDeviation prefers declared-and-failing and confines
withheld-and-skipped to a provider that cannot attempt the behaviour at all — and this provider does
attempt it. Flagging it explicitly because it is the kind of call a reviewer should get to make.

Running it, and the tally

mvn -pl tools/tck -am -DskipTests install     # once
mvn -Ptck -pl providers/ofrep test

A clean run is 65 scenarios: 46 pass, 17 skip, 2 fail. Both failures are the large-integer-flag
fixture gap that flagd-testbed#392 fills —
the mandatory precision scenario and the max-int32 @variants row — not provider defects, and
neither gets a deviation entry.

Worth filing against the Java OFREP provider

  1. No initialize() means a misconfigured provider reports READY. Point it at a bogus baseUrl
    and every evaluation silently returns the code default with GENERAL, while the client status says
    everything is fine. One round trip in initialize() would fix it and unlock both @unavailable
    scenarios.
  2. A value-less success is reported as FLAG_NOT_FOUND — the codeDefaultFlag gap above. This is
    the one with a filed deviation against it.
  3. Not an EventProvider. OFREP's bulk-evaluation endpoint with ETag support is designed for
    exactly this; polling it would unlock @events, @stale and @configuration-change.
  4. ParseError escapes the provider's own error mapping. Resolver.resolve catches only
    GeneralError, but OfrepApi throws ParseError on JsonProcessingException, and the two are
    siblings under OpenFeatureError. The SDK's blanket catch means an application still gets details
    rather than a throw, so the suite does not catch this — but the provider reports PARSE_ERROR where
    its own design intends GENERAL.
  5. Rate-limit backoff is provider-global and cross-flag. A single 429 on one flag sets
    nextAllowedRequestTime, after which every flag's evaluation throws GeneralError until the
    deadline — including flags that were never rate-limited.
  6. Latent, not currently reachable: OfrepResponse.getMetadata() calls ImmutableMap.copyOf on a
    field that stays null when the JSON omits metadata. Every live path launders the object through
    Resolution, whose constructor reads the field directly, so it is safe today — but a direct use of
    a deserialised OfrepResponse would NPE.

@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@aepfli aepfli changed the title feat(provider-tck): adopt the conformance suite in the OFREP provider test(ofrep): run the provider conformance suite against OFREP Aug 24, 2026
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from f51b3f1 to a53854b Compare September 11, 2026 10:36
@aepfli
aepfli changed the base branch from feat/provider-tck to feat/provider-tck-flagd September 11, 2026 10:36
@aepfli aepfli changed the title test(ofrep): run the provider conformance suite against OFREP test(ofrep): adopt the OpenFeature Provider TCK Sep 11, 2026
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from 3180328 to 20cf419 Compare September 11, 2026 10:49
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from a53854b to 36be24c Compare September 11, 2026 10:50
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from 20cf419 to 177ebe0 Compare September 11, 2026 13:44
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from 36be24c to 5be8740 Compare September 11, 2026 13:44
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from 177ebe0 to 5849212 Compare September 11, 2026 16:18
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from 5be8740 to 83d0cc4 Compare September 11, 2026 16:18
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from 5849212 to 76f441a Compare September 11, 2026 17:13
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from 83d0cc4 to acadba1 Compare September 11, 2026 17:13
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from 76f441a to e120ae2 Compare September 11, 2026 17:17
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch 2 times, most recently from 660cc32 to 1036a65 Compare September 12, 2026 06:42
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch 2 times, most recently from 5325453 to 74f2c67 Compare September 12, 2026 10:47
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from 1036a65 to d7a8c3a Compare September 12, 2026 10:47
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from 74f2c67 to 4ba9735 Compare September 12, 2026 13:47
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from d7a8c3a to 27ff382 Compare September 12, 2026 13:47
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from fdbe6a8 to 8ac8f62 Compare September 13, 2026 08:48
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from c826be5 to 08fa51a Compare September 13, 2026 08:48
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from 8ac8f62 to a0bd1eb Compare September 13, 2026 09:09
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from 08fa51a to 94d2c35 Compare September 13, 2026 09:09
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from a0bd1eb to c7c4569 Compare September 13, 2026 14:48
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from 94d2c35 to a447b4c Compare September 13, 2026 14:48
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from c7c4569 to d7a1c24 Compare September 13, 2026 18:03
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from a447b4c to 435edfb Compare September 13, 2026 18:03
@aepfli
aepfli force-pushed the feat/provider-tck-flagd branch from d7a1c24 to 1e3c1dc Compare September 13, 2026 19:22
@aepfli
aepfli force-pushed the feat/provider-tck-ofrep branch from 435edfb to b001ced Compare September 13, 2026 19:23
OFREP is a protocol, not a vendor, so the suite needs no new infrastructure:
flagd already serves the OFREP HTTP API on 8016 inside the flagd-testbed image
that the flagd TCK suites use, alongside the launchpad control API on 8080. The
Compose stack is therefore the same image with a different port exposed, and the
whole adoption is one test class plus one dependency.

Four capabilities are withheld, all traceable to the same fact: OfrepProvider
implements FeatureProvider rather than extending EventProvider and overrides no
lifecycle method, so it has no state, no stream, no poll loop and no
initialize(). It cannot emit events (EVENTS), cannot observe the backend going
away (STALE) or changing (CONFIGURATION_CHANGE), and cannot fail initialisation
against a dead port (UNAVAILABLE_INIT). Each omission is justified against
specific lines of the provider in the capabilities() javadoc. events.feature and
lifecycle.feature are both tagged @events at feature level, so 5 scenarios are
reported as skipped and 24 run.

OBJECT and STRICT_NUMERIC_TYPING are both declared. Unlike the flagd provider,
OFREP does not silently narrow a float to an integer: values are deserialised by
a plain Jackson ObjectMapper into an untyped Object, so a JSON fraction arrives
as Double and a JSON integer as Integer, and handleResolved admits a value only
on an exact type.isInstance check. float-flag requested as an integer is
reported as TYPE_MISMATCH with the code default rather than truncated to 0.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The base renamed the containerised base class, retired
@strict-numeric-typing in favour of @numeric-coercion, and now refuses a
declaration that names a reserved or not-applicable capability.
EnumSet.complementOf swept up @large-integers, @targeting and @caching,
so the suite would have stopped at startup; Capability.declarableExcept
leaves those out on its own.

LIFECYCLE is withheld as well, which the complement had quietly claimed
since the capability appeared: OfrepProvider has no initialize(), so the
readiness scenario passed exactly as it does for NoOpProvider, and the
new shutdown scenarios gated by the same tag are skipped rather than
passed vacuously. NUMERIC_COERCION is withheld because the tag now
requires the lossless direction too, and Resolver.handleResolved admits a
value only on an exact type.isInstance check: integer-flag requested as
a float arrives from Jackson as an Integer and is refused. Strict typing
in both directions is a choice under the suite's model, not a defect, so
no KnownDeviation goes with it. Read from the source, not from a run.

The class Javadoc records that flagd-testbed v3.8.0 serves none of the
six new canonical flags, so the four untagged scenarios that read them
fail FLAG_NOT_FOUND until the testbed is updated; that is the stack's
gap, not the provider's, so it is documented rather than declared.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…tbed lacks

The same note the flagd adoption carries, next to the image tag it is
about: flagd-testbed v3.8.0 serves none of the six flags the bumped
assets added, so the untagged scenarios that read them fail
FLAG_NOT_FOUND until the testbed is updated, and the tag here is what to
bump when it is.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…vidence

The falsy-flag rename in the base removes three failures here without touching
the provider: flagd-testbed already served boolean-zero-flag, integer-zero-flag
and string-zero-flag with zero/non-zero variants, and spec ba002ce8 moved the
canonical names onto the testbed's rather than the other way round. The suite
goes from four untagged failures to one. The notes now say what is actually
missing -- large-integer-flag, and only that, because huge-integer-flag sits
behind @large-integers and integral-float-flag behind @numeric-coercion, neither
of which this provider declares.

@numeric-coercion stays withheld, now on stated evidence rather than a shorter
argument. Go's OFREP provider declares it and this one does not, which looked
like an unexamined declaration on one side; it is not. handleResolved admits a
value only on an exact type.isInstance check (Resolver.java:183-191) with no
integral check and no round trip anywhere in the path, so of the tag's three
scenarios this provider passes one: the lossy case is right for the wrong reason,
and both lossless cases fail, integer-flag requested as a float and
integral-float-flag requested as an integer alike. Declaring it would turn two
scenarios red -- three, counting that the testbed cannot serve
integral-float-flag at all. Go coerces and this does not; the two declarations
describe two implementations, not one protocol, which is possible precisely
because OFREP is JSON and integer-ness is the provider's decision.

No KnownDeviation accompanies it, and that is a decision rather than silence.
Appendix F is explicit that this is the one capability the specification does not
define, that its rule is borrowed from flagd's ADR, and that "a provider that
behaves differently is not violating the specification" -- having retracted an
earlier draft that called non-declaration an admission of a known bug. A
deviation entry would assert a defect the spec says is not one: the opposite
mistake from a vacuous declaration, in the same currency. What the entry does
record is that the reasoning is source-derived, that no unit test pins the
numeric pair, and what a run would have to show for the declaration to change.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The base gates "A provider that was shut down can be initialized again" on
@reinitialization, because Requirement 2.5.2 permits reuse after shutdown rather
than requiring it. declarableExcept(...) hands the new tag out by default, so a
provider that cannot be restarted has to say so or it publishes a claim nothing
examined -- the one failure mode this declaration exists to prevent.

OfrepProvider cannot be restarted. shutdown() terminates the executor the HTTP
client runs on (OfrepProvider.java:90-108) and the class overrides no
initialize(), so nothing recreates it. That is the choice 2.5.2 offers rather
than a defect, and no KnownDeviation accompanies the omission.

It changes no result on its own: the scenario carries @lifecycle too, which this
provider already withholds because it has no initialisation to observe, so the
skip was happening either way. The declaration is what stops being a
half-truth. The @lifecycle bullet no longer counts re-initialisation among the
scenarios that tag gates, since it does not any more.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The TCK no longer sets @large-integers apart as "not applicable in Java", so
declarableExcept(...) no longer leaves it out on its own and this suite names it
alongside the six capabilities it already withholds.

The run is unchanged: the scenario was skipped before and is skipped now, because
the SDK's integer accessor is a 32-bit Integer and 2^53 - 1 has no room in it. That
is a fact about the SDK rather than about OFREP, which is why it is recorded once in
Appendix F and needs no knownDeviations entry here -- unlike the numeric-coercion
gap, which is the provider's own.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
@Variants and @targeting arrived with the base's submodule bump and
declarableExcept picks both up, so the declaration grew by two capabilities
while the javadoc that argues every withheld tag at length said nothing about
either. One run against flagd-testbed v3.8.0 through the OFREP HTTP API: 38
pass, 12 skipped, 2 failed.

@targeting is worth more here than its name suggests. The provider sends the
evaluation context in the request body and the backend evaluates the rule, so
targeting-key-flag's three scenarios are the only ones in the canonical set that
would notice a context dropped on the way out -- every other flag resolves the
same way with or without one. All three pass.

@Variants passes seven of its eight rows. The eighth asks for
large-integer-flag's max-int32 and is answered with no variant, because v3.8.0
does not serve that flag at all -- the gap the Compose header already records
for the untagged precision scenario, now reached twice rather than once, so
"one untagged scenario" is a scenario short. Withholding the tag would hide
both failures behind a claim about the provider that the run does not support.

The complementOf paragraph called @targeting reserved alongside @caching. It is
declarable now; the argument survives with one tag, because what makes the form
of the call right is not how large the reserved set happens to be.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Measured, not assumed, and the measurement overturned the expectation it
was made to confirm.

Declared, all four rows of the new outline fail: 56 scenarios, 38
passing, 12 skipped, 6 failed, the four extra failures beyond the two
testbed ones all reading "expected: null but was: FLAG_NOT_FOUND". The
value assertion passes, because the provider returns the code default on
an error -- right answer, wrong reason.

The expectation was that this is architectural: the caller's default never
leaves the process, so a backend cannot return it. That is not what the
backend does. flagd's OFREP endpoint answers a disabled flag with 200,
reason DISABLED and no value member, probed directly against the pinned
v3.8.0 image. OFREP's evaluationSuccess requires only key and reason;
value is not required, because one shape a success may take is
codeDefaultFlag -- "This schema has no value property. The provider must
use the code default value when processing this response." DISABLED is in
the reason enum. The response is well-formed and says exactly what the
scenario asserts.

So this is a provider gap against a MUST in the protocol the provider
implements, and it is wider than the rows that found it: every
codeDefaultFlag response reaches the application as FLAG_NOT_FOUND, so an
application checking the error code sees a failure on an evaluation that
succeeded. handleResolved treats a null value as an absent flag and
discards the reason it parsed a field earlier.

The tag stays withheld, because it fails and a conformance run must not
pass it. But it carries a KnownDeviation, which is the opposite call from
@numeric-coercion next to it, and the difference is where the rule lives:
numeric coercion is Appendix F borrowing flagd's ADR with no
specification behind it, whereas codeDefaultFlag is normative OFREP. A
bare omission would read as the same kind of choice, and this is not a
choice. Untracked for now; delete both once handleResolved honours a
value-less success.

Nothing is owed upstream. The four disabled-* flags are served by the
image already pinned, which the Compose header now records so the
failures are not mistaken for another testbed gap.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Import-path and coordinate churn only: the artifact is
dev.openfeature.contrib.tools:tck, the version range starts at 0.1.0, and
the four imports come from dev.openfeature.contrib.tools.tck. No
behavioural change, and no change to what is declared or withheld.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…tainers

Two fixes to this module's POM, both of which were latent.

testExclusions was missing. The parent POM feeds <exclude>${testExclusions}</exclude>
to Surefire but defines no default, so a module that wants the Docker gate has
to declare the property and a module that forgets one runs a Docker-dependent
suite in every job. providers/flagd declares it; this module did not, so
OfrepTckTest started a Compose stack during plain `mvn verify` and ended
"Tests run: 56, Failures: 2" on the recorded testbed gaps. So the ofrep PR was
red in CI while the flagd PR was green for the opposite reason - one ran a suite
it should gate, the other gated a suite nobody ran. Adding the exclusion makes
the policy uniform: `mvn -Pcodequality -pl providers/ofrep -am verify` is now
BUILD SUCCESS with 18 tests and no Compose stack.

Testcontainers is now declared here. The TCK moved it to provided/optional so
that the majority of adopters, which have no backend and never load
ContainerizedProviderTckTest, stop resolving it; a containerised adopter brings
its own instead. A previous pass asserted that both adoptions in this repository
already did - that was true of providers/flagd and false of this module, which
was relying on the transitive edge. Stated plainly because the assertion was
wrong, not because the fix is interesting.

Verified with the suite actually run: 56 scenarios, 38 passed, 16 skipped for
withheld capabilities, 2 failed on the flags flagd-testbed v3.8.0 does not
serve - unchanged, so the dependency now resolves from here rather than through
the TCK.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The other half of adding testExclusions. An exclusion nobody writes down is
indistinguishable from an oversight, and that is not a hypothetical here: this
module's exclusion was missing precisely because nothing said the convention
existed.

So the README states the policy and its reasoning - Docker-gated, excluded from
every job that exists, run by hand by a maintainer before merging a change to
resolution or error behaviour, with the result quoted in the pull request. A
scheduled or path-filtered workflow was considered and declined. It also gives
the command and says which failures are expected, so a reader can tell a
regression from the recorded testbed gaps, and notes that several capabilities
are withheld because OFREP puts the decision on the server rather than in the
provider - a fact about the protocol, not a defect.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Appendix F now carries "Running the suite in CI", promoted there because the
same reasoning restated in four adoption READMEs is where it drifted. So this
section keeps the mechanism and the local record -- that this module had no
testExclusions at all until it was added, which is the appendix's
"exclusion nobody wrote down" -- and links to the appendix for the argument
rather than paraphrasing it.

Also states that no profile in this module touches the property, resolved rather
than read:

  mvn -Pe2e -pl providers/ofrep help:evaluate -Dexpression=testExclusions
      -> **/e2e/*.java
  mvn       -pl providers/ofrep help:evaluate -Dexpression=testExclusions
      -> **/e2e/*.java

Unlike providers/flagd, which needs an e2e profile for its legacy suites and
therefore narrows the pattern instead of clearing it.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
This suite declares everything declarable except nine capabilities, so it picked
@standard-reasons up by default the moment the TCK gained it. Measured before it
was written down.

Eight of reason.feature's nine scenarios run and pass: STATIC for the four
rule-less flags, ERROR beside FLAG_NOT_FOUND and TYPE_MISMATCH, and -- because
@targeting is declared here -- TARGETING_MATCH and DEFAULT either side of
targeting-key-flag's rule. The ninth carries @disabled-flags as well and is
skipped for that omission, which is the right outcome rather than a second report
of the same gap: what this provider does wrong with a value-less success is
already stated once, in the withheld capability and its KnownDeviation, and a
reason it never reaches is not more evidence of it. So the tag means "the
standard vocabulary, over the responses this provider actually completes", and a
reader sees the withheld @disabled-flags beside it and can tell which scenario
went unasked.

A clean run is 65 scenarios, 46 passing, 17 skipped and 2 failing, up from 56, 38
and 16. The two failures are the same testbed gaps as before.

Also records something this pass measured rather than introduced: the suite is
intermittently flaky. About half of the runs carry one or two extra failures
where an evaluation comes back as the code default, or as FLAG_NOT_FOUND where
TYPE_MISMATCH was expected, or with reason ERROR where a resolution was expected.
The victim moves between errors.feature, evaluation.feature and reason.feature,
so it is not a property of any assertion. Eight runs were measured, five at this
revision and three at ccdb8879, and the old pin produced a seven-failure run and
a two-failure run from the same tree -- so this predates the reason scenarios and
is not caused by them.

That is the flagd-testbed readiness window of open-feature/flagd-testbed#394
reaching a provider that holds nothing between calls, so every evaluation races
the stack afresh. Recorded in the class javadoc and the README with an explicit
instruction not to cover it with a settle after control calls, because a suite
that sleeps instead of holding the control API to its promise stops being able to
detect when the promise breaks.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
This suite's declarableExcept list is long, and every name in it is something this
provider genuinely cannot do -- no lifecycle to observe, no events, no connection
to lose, a backend that decides so the caller's default never leaves the process,
a deserialiser that keeps the numeric types strictly apart. @large-integers was
the odd one out: it said nothing about OFREP at all, only that
Client.getIntegerDetails is a 32-bit Integer. The TCK refuses it centrally now, so
it is gone from the list and from the paragraph that explained it.

That matters more here than in the flagd suite, because here it was one name among
nine and a reader had no way to tell which of the nine described the provider.
Now all of them do.

Measured on the pinned testbed image: 65 scenarios, 46 passed, 17 skipped, 2
failed -- a clean run, identical to the previous pass, since this changes the
reason for a skip rather than the count. The stream now carries four "provider
does not declare capability" reasons and one "the Java SDK cannot express
capability LARGE_INTEGERS", which is the distinction a reader of the report needs.

No KnownDeviation changes: the existing DISABLED_FLAGS entry is unaffected, and a
capability no Java provider can be asked was never a deviation to record.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…ide it

Two capability omissions in this suite are argued in the javadoc, and Appendix F's
declaring rules have since drawn lines through both arguments.

@numeric-coercion is withheld because the provider does not coerce -- it
type-checks in both directions -- and no requirement says it must, which is the
question that comes first and settles this one. The paragraph that follows also
observed that declaring the tag would run a scenario the pinned testbed cannot
serve. That is now explicitly not a reason: a scenario failing for a missing
fixture is not a provider defect, so it argues for nothing. It stays as a note
about what such a run would look like, marked as one. The appendix's scenario-level
rule does not reach this omission either, and the javadoc says why rather than
leaving a reader to wonder: that rule is about whether a question is askable, and
all three of these are -- what it does not decide is whether an answer is owed.

@disabled-flags is the opposite case and the javadoc now says so plainly. The
provider receives the codeDefaultFlag response, parses its reason and answers with
the wrong error code, so it does attempt the behaviour, and the settled guidance
prefers declaring the tag and letting the four rows fail with this same deviation
beside them. That is a change of results rather than of prose and is not made
here; recording it is what stops it being lost.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The same shape providers/flagd just gained, and for the same reason Appendix F
gives: a conformance run carries failures by design wherever OfrepTckTest
declares a knownDeviation, so a signal it shares with a suite that is expected
green ends with somebody silencing the informative half.

The `tck` profile clears this module's **/e2e/*.java exclusion and narrows
Surefire's includes to **/e2e/*TckTest.java in the same breath, so
`mvn -Ptck -pl providers/ofrep test` runs OfrepTckTest and nothing else -
65 scenarios, 46 passing, 17 skipped and 2 failing, which is the clean run the
README already documents.

This module had no profiles at all, so the README's claim that no profile
touched testExclusions was true and now is not; it names the one that does.
Nothing activates `tck` in CI.

The README says not to add `-am` to that run, and why: it pulls tools/tck into
the reactor and runs its 246 tests first, so a failure there comes out as a
`-Ptck` failure. A one-off `install` is what `-am` was there for.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The suite lived in an `e2e` package it was the only member of, and every
selector reached it by filename: `**/e2e/*.java` to exclude it,
`**/e2e/*TckTest.java` to run it. It now lives in `src/test/java/.../ofrep/tck/`
and the selectors name that directory. The `e2e` package is gone -- there was
never an end-to-end suite in this module for the adoption to sit beside, which
is the clearest form of the argument: it was filed under a category it was not.

The class drops what the directory now says. `OfrepTckTest` in package
...ofrep.tck said "tck" twice; it is `OfrepTest`. The `*Test` suffix stays,
because Surefire's default includes need it -- that is a different thing from
the selector being removed. The name a run is filed under is unchanged:
configuration() strips the JUnit suffix either way, so it was and remains
"ofrep".

Both halves of the `tck` profile are still needed, for the reason they always
were: the include alone leaves the exclusion in force and runs nothing, and
dropping the exclusion alone runs this module's unit tests alongside the suite.
testExclusions is still a Surefire and not a compiler exclusion, so the suite
still compiles in the default build.

The tally does not move: 65 scenarios, 46 passed, 17 skipped, 2 failed. Same
scenarios, same results, a different directory.

The profile comment documenting the run had kept `-am` on it, which the README
already warns against; it says the command that works now.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants