feat: Add the FDv2 data system orchestrator - #190
Conversation
4ea2eb8 to
775e611
Compare
775e611 to
2fa17de
Compare
2fa17de to
a523fa8
Compare
a523fa8 to
3e77e84
Compare
3e77e84 to
f3e5a99
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f3e5a99. Configure here.
…they are consumed
f3e5a99 to
58c2a46
Compare
keelerm84
left a comment
There was a problem hiding this comment.
A few questions and suggestions on the orchestrator, mostly around the initializer chain and test coverage.
| init_complete: Arc<dyn Fn(bool) + Send + Sync>, | ||
| shutdown_receiver: broadcast::Receiver<()>, | ||
| ) { | ||
| let initializers = self |
There was a problem hiding this comment.
If the first initializer returns a basis, then we wouldn't need to build the subsequent ones. Maybe we should build them as we iterate instead of all at once then?
| .states | ||
| .iter() | ||
| .position(|s| *s == SourceState::Available); | ||
| matches!((first, self.current_factory_index), (Some(f), Some(c)) if f == c) |
There was a problem hiding this comment.
Can this not be simplified down first == self.current_factory_index && first.is_some()?
| _ = shutdown => return, | ||
| event = initializer.run().fuse() => { | ||
| if let FDv2SourceResult::ChangeSet(change_set) = event.result { | ||
| if !matches!(change_set.kind, ChangeSetKind::None) { |
There was a problem hiding this comment.
Initializers can arrive with a full change set but no basis (e.g. file based initialization). So applying a change set and determining if the init chain is done are two different things.
| } | ||
| } | ||
|
|
||
| struct MockInitializer { |
There was a problem hiding this comment.
Might be worth implementing the test data source for FDv2 so future tests can rely on that if needed.
| } | ||
|
|
||
| #[tokio::test] | ||
| async fn initializer_basis_signals_once_and_propagates_selector() { |
There was a problem hiding this comment.
Based on my earlier comment, you will also want a test that gets a payload from the first source, doesn't have a basis, and then continues onto the second one.

Summary
Adds the FDv2 orchestrator, the
DataSystemimplementation that owns thein-memory store and keeps it populated. It runs an initializer phase to obtain
a basis, then a synchronizer phase for ongoing changes.
The initializer phase tries each initializer in order until one produces a
basis. The synchronizer phase then runs one synchronizer at a time and rotates
between them on three triggers:
interruption.
interval.
Initialization is signaled as complete on the first basis and as failed once
every source is exhausted without one, because the client's initialization path
already resolves to success or failure. Synchronizer status transitions are
logged, with interruptions at info (not repeated while ongoing) and terminal
errors at warn.
Note
Overview
Adds
FDv2DataSystem, the FDv2DataSystemimplementation that owns an in-memory store and keeps it updated via a tokiorunloop.Initializer phase: tries each initializer factory in order until one returns a non-
Nonechangeset; applies it withTransactionalDataStore::apply, signalsinit_complete(true), and tracks the changeset selector for later sync calls.Synchronizer phase:
SourceManagerrotates among synchronizer factories—cyclically skipping blocked sources, with fallback after sustainedInterrupted(timer), recovery back to the prime when not on primary, and block + advance onTerminalError. Successful changesets update the store and selector;ChangeSetKind::Nonedoes not clobber the selector. Shutdown ends the loop without init signaling; if no basis was ever obtained,init_complete(false)runs once.Wires the module in
fdv2/mod.rsand removes staledead_codeallows onChangeSet.selectorandTransactionalDataStore. Includes broad unit/integration-style tests for rotation, timers, init signaling, and selector propagation.Reviewed by Cursor Bugbot for commit ffcc8cf. Bugbot is set up for automated code reviews on this repo. Configure here.