Skip to content

[FFL-2785] Reduce flags provider initialization latency - #3078

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 39 commits into
developfrom
sameerank/FFL-2785/avoid-blocking-provider-init
Oct 21, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 39 commits into
developfrom
sameerank/FFL-2785/avoid-blocking-provider-init

Conversation

@sameerank

@sameerank sameerank commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor

What and why?

Reduce flag-provider startup latency by removing unnecessary waits around the assignment request. Today, setProviderAndWait() waits for the initial cache read before fetching, runs request setup on Core's shared context queue, and waits for cache encoding/persistence setup before reporting .ready.

Demonstrated benefit: in a controlled Simulator test with a mocked response and a two-second storage-queue backlog, median repository initialization dropped from 2,041 ms to 0.21 ms across ten samples per version. Every PR sample completed before the cache read finished. This confirms removal of the cache dependency, not end-to-end setProviderAndWait() or app-startup latency.

How?

  • Fetch alongside the initial cache read and publish successful assignments before cache encoding/persistence.
  • Use separate serial flags queues for request/response processing and cache encoding, at .utility QoS. This isolates work without raising its priority above other SDK features.
  • Preserve matching-cache fallback on failure and prevent older reads or requests from overwriting newer assignments or client state. Callbacks and listeners stay outside locks.

Allowing cache reads, fetches, and persistence to overlap requires explicit ordering guards so late work cannot overwrite newer assignments or undo reset.

Changes are confined to DatadogFlags; no public API, Core queue priority, or network timeout changes. The required Core context snapshot can still wait behind shared work, tracked separately in FFL-3328. If startup remains slow, timing logs should confirm whether this remaining wait is responsible before pursuing that follow-up.

Suggested review order

  1. FlagAssignmentsFetcher.swift: request/response queue isolation and completion delivery.
  2. FlagsRepository.swift: overlapping fetch/cache work, matching-cache fallback, result ordering, and persistence.
  3. FlagsStateManager.swift: state updates under lock, with superseded notifications skipped outside the lock.
  4. Tests: start with the repository's Initialization Latency section and the fetcher's queue-isolation test, then review cache, lifecycle, and overlapping-request coverage.
Supporting diagnostics, cache behavior, and validation

Customer Evidence
Separate customer debugging runs showed approximately 10.5 seconds before fetching and 8.9 seconds between fetch success and .ready. These were observed delays, not artificially injected; the competing work responsible was not confirmed.

Controlled Benchmark
October 20, 2026: comparing develop at 176506095 with this PR at 215c81f.

Scenario Develop median PR median
No backlog 0.53 ms 0.27 ms
Two-second storage backlog 2,041 ms 0.21 ms
Storage backlog, obfuscated flags 2,041 ms 0.38 ms
Two-second Core context backlog 2,041 ms 2,041 ms

Ten samples per scenario and version, excluding one warmup. Measured repository construction through context-update completion on the Simulator, using real Core queues and mocked network responses.

Every storage-blocked PR sample completed before the cache read finished. The required Core context-snapshot wait remains. These results demonstrate removal of a queue dependency, not expected customer startup times.

Behavior Preserved

  • Matching cached flags remain available during reconciliation. A failed fetch waits for the initial cache read, then selects matching cache (.stale) or .error.
  • The first initialization retains its configurable timeout: five seconds by default, disabled with nil. Later updates have no initialization timeout; timing out does not cancel the fetch.
  • Ordering guards protect newer assignments and resets from late results. Superseded requests still complete their callbacks, and flushing drains pending cache writes.

Validation
All 200 Flags tests passed, including cache/concurrency regressions and flag-key obfuscation coverage. Targeted SwiftLint also passed.

Tests and benchmarks ran with Xcode 26.2, iPhone 17 / iOS 26.2 Simulator, Debug, on an Apple M3 Max. Both used SWIFT_VERSION=5 because existing DatadogTrace/OpenTelemetry errors blocked the default Swift 6 build locally.

Review checklist

  • Feature or bugfix MUST have appropriate tests (unit, integration)
  • Make sure each commit and the PR mention the Issue number or JIRA reference
  • Add CHANGELOG entry for user facing changes
  • Add Objective-C interface for public APIs - see our guidelines (internal)
  • Run make api-surface when adding new APIs

No public or Objective-C API changes; the last two checklist items are not applicable.

Metadata

Changelog:

Label Message
FIX Start the DatadogFlags assignments request without waiting for the initial cache read, and report successful assignments before cache persistence.
IMPROVEMENT Overlapping FlagsClient.setEvaluationContext calls no longer let an earlier result replace assignments for a newer context.

@sameerank
sameerank force-pushed the sameerank/FFL-2785/avoid-blocking-provider-init branch from 9ea3a06 to 31463c3 Compare July 21, 2026 01:22
@sameerank
sameerank marked this pull request as ready for review July 21, 2026 21:41
@sameerank
sameerank requested review from a team as code owners July 21, 2026 21:41
@sameerank
sameerank requested review from pavlokhrebto and typotter and removed request for a team July 21, 2026 21:41

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

Comprehensive state machine validation completed. The PR successfully refactors FlagsRepository to avoid blocking provider initialization on the initial disk cache read by implementing an asynchronous callback-based approach and consolidating state under atomic locks. Version-based race prevention, disk-cache separation, and context-matching logic are sound. All critical timing variations are handled correctly by the test suite.

Was this helpful? React 👍 or 👎

📊 Validated against 12 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit 58cb7b8 · What is Autotest? · Any feedback? Reach out in #autotest

@sbarrio
sbarrio requested a review from tienquocbui July 22, 2026 07:04
leoromanovsky
leoromanovsky previously approved these changes Jul 22, 2026

@leoromanovsky leoromanovsky left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran locally; yes, I believe this will fix the customer issue in the happy path.

My reading, though, is that if there is a network outage, we'll still see a long startup delay. I think it lacks an assertion that initialization has an upper latency bound when both cache and network paths are unhealthy.

@sameerank
sameerank force-pushed the sameerank/FFL-2785/avoid-blocking-provider-init branch from e6d98c6 to 4e24256 Compare July 30, 2026 08:39

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e24256a3a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 97c2955afc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 91ee41044b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c05b791191

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread CHANGELOG.md Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 502dbbd6d6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants