Skip to content

Use canonical FFE fixtures#11355

Open
leoromanovsky wants to merge 7 commits into
masterfrom
leo.romanovsky/ffe-canonical-fixtures-20260512
Open

Use canonical FFE fixtures#11355
leoromanovsky wants to merge 7 commits into
masterfrom
leo.romanovsky/ffe-canonical-fixtures-20260512

Conversation

@leoromanovsky
Copy link
Copy Markdown
Contributor

@leoromanovsky leoromanovsky commented May 12, 2026

Motivation

Make DataDog/ffe-system-test-data the canonical source for FFE/OpenFeature JSON fixtures in dd-trace-java. This removes copied smoke-test fixtures and moves evaluator behavior coverage to the shared corpus. Depends on DataDog/ffe-system-test-data#9.

Changes

  • Add dd-smoke-tests/openfeature/src/test/resources/ffe-system-test-data as a git submodule pinned to the canonical fixture update.
  • Remove copied OpenFeature fixture JSON resources.
  • Update the smoke test to load canonical ufc-config.json and all sorted evaluation-cases/*.json files.
  • Assert canonical result values and reasons from the JSON corpus.
  • Remove the stale programmatic evaluator fixture set in favor of the canonical JSON-driven loop.
  • Fix the OpenFeature smoke CodeNarc violation by removing an unnecessary Groovy import.
  • Add weekly Dependabot gitsubmodule updates.

📖 https://datadoghq.atlassian.net/wiki/spaces/PANA/pages/6713639211/FFE+SDK+Fixture+Contribution+Guide

Decisions

  • Shared evaluator behavior must come from JSON fixtures in ffe-system-test-data, not programmatic or copied fixture sets.
  • dd-trace-java already supports submodules and recursive checkout in CI; this PR uses the same pattern for the OpenFeature fixture corpus.
  • PHP is intentionally excluded until its OpenFeature client lands.

Validation

  • JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home PATH=/opt/homebrew/opt/openjdk@17/bin:$PATH ./gradlew :dd-smoke-tests:openfeature:codenarcTest
  • JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home PATH=/opt/homebrew/opt/openjdk@17/bin:$PATH ./gradlew :products:feature-flagging:feature-flagging-api:test --tests datadog.trace.api.openfeature.DDEvaluatorTest

@leoromanovsky leoromanovsky marked this pull request as ready for review May 18, 2026 12:21
@leoromanovsky leoromanovsky requested review from a team as code owners May 18, 2026 12:21
@leoromanovsky leoromanovsky requested review from dougqh, sameerank and typotter and removed request for a team May 18, 2026 12:21
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 18, 2026

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 057cdefcb8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +94 to +97
return ProviderEvaluation.<T>builder()
.value(defaultValue)
.reason(Reason.DEFAULT.name())
.build();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore missing-flag error reporting

When a configured provider is asked for a flag key that is absent from config.flags, this now returns a normal DEFAULT evaluation with no FLAG_NOT_FOUND error. In that scenario OpenFeature callers and the existing FlagEvalHook metrics path will report the lookup as a successful default rather than a missing-flag error, so dashboards/alerts that depend on error.type=flag_not_found stop seeing these failures. If the intent is only to return the supplied default value, keep that value but preserve the error reason/code for absent flags.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant