Skip to content

onboarding: data-plane picker in the signup form - #2045

Open
SeanWhelan wants to merge 13 commits into
mainfrom
sean/onboarding-data-plane-picker
Open

SeanWhelan wants to merge 13 commits into
mainfrom
sean/onboarding-data-plane-picker

Conversation

@SeanWhelan

@SeanWhelan SeanWhelan commented Aug 5, 2026 •

Copy link
Copy Markdown
Contributor

Issues

estuary/sre#29 (Phase 2 — tracked in a roadmap doc, not an estuary/ui issue)

Changes

sre#29

  • Adds a DataPlaneSelector to the signup form (BetaOnboard.tsx), between the organization name field and the survey. It's backed by the existing unauthenticated publicDataPlanes query, which this PR consumes for the first time — the authenticated dataPlanes query returns nothing for a brand-new signup, since there are no grants yet.
  • Auto-selects the newest open plane in AWS us-east-1, falling back to the first plane otherwise, so submitting without touching the picker still records an explicit, valid choice. It picks by region rather than a hardcoded plane name, and publicDataPlanes only returns open planes, so replacing a plane (open c2, close c1) needs no UI change. While c1 and c2 are both open, the picker lists both and preselects c2.
  • Fails safe: if the plane list can't load, the picker renders nothing and the claim just omits requestedDataPlane rather than sending it as an explicit null, so the backend falls back to its own default instead of the signup form breaking. The failure is sent to PostHog as an Onboarding:DataPlanes event, so we can see when it happens.
  • Extends the betaOnboard directive's generateUserClaim to forward requestedDataPlane only when it's set, so backends that don't know the field yet aren't affected.
  • Adds a usePublicDataPlanes() hook and PublicDataPlaneNode type, following the existing useDataPlanes()/DataPlaneNode pattern already in the codebase.
  • The selected plane is kept in useState in BetaOnboard and passed down to the picker, rather than added to the onboarding Zustand store.

Options are grouped by cloud provider and labelled with the region plus the cluster suffix (e.g. us-east-1 c1). The suffix is what tells apart two public planes in the same region (already the case for GCP). If a name doesn't parse, the label falls back to the full catalog name. The full name is always what gets submitted. The picker can't be cleared, since a plane is always preselected.

This depends on estuary/flow#3318 merging and deploying first. The agent rejects unknown claim fields, so if this ships before that one is live, every signup fails with invalidClaims. Merging flow isn't sufficient on its own — an old agent still running will reject the claim.

Tests

Manually tested

I ran a full signup through the real onboarding form against a local flow stack. The picker listed the seeded public planes grouped by cloud provider, auto-selected the platform default, and a completed registration produced the correct storage_mappings row with the picked plane recorded as the tenant's default.

I repeated this against a synthetic AWS-shaped plane with COLOCATED_TRIAL_BUCKETS enabled on the backend. The resulting storage mapping showed the derived S3 bucket, confirming the picker's choice flows through correctly end to end.

I also checked what happens against a real, older backend that predates this feature: the query fails with a clear GraphQL error, the picker just doesn't render, and the rest of the form stays fully usable. No crash.

Finally, with the companion PR's migration applied, a data plane marked closed disappears from the picker's options — the list and the backend agree on what's selectable.

Automated tests

tsc --noEmit and npm run lint both pass clean. I didn't add new unit tests since this is UI wiring around an already-tested backend contract; flagging that for reviewer input rather than assuming it's fine.

Playwright tests ran locally

  • Admin
  • Captures
  • Collections
  • HomePage
  • Login
  • Materialization

Screenshots

picker-collapsed picker-open

@github-actions

github-actions Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

⚪ Code Health

No change to the dead-code surface.

48 Unused files

File imported nowhere — delete (or import) it.

     src/hooks/useDelay.ts
     src/hooks/useDraft.ts
     src/pages/NoGrants.tsx
     src/pages/OAuth.tsx
     src/services/encryption.ts
     src/types/global.ts
     src/types/vitest.ts
     src/components/graphs/TaskHoursByMonthGraph.tsx
     src/components/tables/Link.tsx
     src/context/LoopIndex/index.tsx
…and 38 more

66 Unused exports

Exported symbol with no references outside its own file — un-export it, or delete it if unused entirely

     src/context/Theme.tsx : logoColors
     src/context/Theme.tsx : intensifiedOutlineThick
     src/context/Theme.tsx : tableAlternateRowsSx
     src/context/Theme.tsx : draggableChipIconSx
     src/context/Theme.tsx : hiddenButAccessibleInput
     src/context/Theme.tsx : primaryColoredBackground_hovered
     src/context/Theme.tsx : detailsPanelBgColor
     src/context/Theme.tsx : menuBackgroundColor
     src/context/Theme.tsx : flexGrowToSiblingsSx
     src/context/Theme.tsx : shardTableRow
…and 56 more

29 Unused exported types

Exported type with no references outside its own file — un-export it, or delete it if unused entirely

     src/utils/billing-utils.ts : FREE_GB_BY_TIER
     src/types/index.ts : InferredSchemas
     src/types/index.ts : Shard
     src/components/shared/WizardDialog/index.ts : WizardStep
     src/hooks/forks/react-use-oauth2/components/use-oauth2.ts : State
     src/api/dataPlanes.ts : AwsDnsEntry
     src/stores/ShardDetail/types.ts : TaskShardDetailsWithShard
     src/stores/ShardDetail/types.ts : ShardDetails
     src/components/tables/Logs/types.ts : RefreshLogsFunction
     src/types/schemaModels.ts : CollectionSchema
…and 19 more

14 Unused exported enum members

An enum member referenced nowhere

     src/services/supabase.ts : CONNECTOR_TAGS
     src/services/supabase.ts : DRAFTS_EXT
     src/services/supabase.ts : TASKS_BY_DAY
     src/stores/Tables/hooks.ts : accessGrants
     src/stores/Tables/hooks.ts : accessLinks
     src/stores/Tables/hooks.ts : billing
     src/stores/Tables/hooks.ts : connectors
     src/stores/Tables/hooks.ts : entitySelector
     src/stores/Tables/hooks.ts : prefixes
     src/stores/Tables/hooks.ts : prefixAlerts
…and 4 more

5 Unused dependencies

In package.json but never imported

     package.json : @mui/lab
     package.json : @testing-library/jest-dom
     package.json : @urql/exchange-retry
     package.json : logrocket-react
     package.json : stripe

3 Unused devDependencies

In package.json devDependencies but never used

     package.json : @types/logrocket-react
     package.json : @types/react-inspector
     package.json : sharp

@SeanWhelan
SeanWhelan requested a review from GregorShear August 7, 2026 10:25
@SeanWhelan
SeanWhelan force-pushed the sean/onboarding-data-plane-picker branch from c1f79e9 to 7020609 Compare August 26, 2026 17:25
@SeanWhelan
SeanWhelan marked this pull request as ready for review August 26, 2026 17:31
@SeanWhelan
SeanWhelan requested a review from a team as a code owner August 26, 2026 17:31
…onboarding

The authenticated dataPlanes query returns nothing for a brand-new signup
(no grants yet), so the onboarding plane picker needs the control-plane
API's unauthenticated publicDataPlanes query instead.
…rd claims

Adds a plane picker between the organization name field and the survey,
backed by the unauthenticated publicDataPlanes query (a brand-new signup
has no grants yet, so the authenticated dataPlanes query returns nothing).
Auto-preselects aws-us-east-1-c1 (falling back to the first available
plane) so submitting without touching the picker still records an
explicit, valid choice; fails safe to no picker + an omitted claim if the
plane list can't be loaded, letting the backend apply its own default.

generateUserClaim now forwards requestedDataPlane only when set, never as
an explicit null, so older backends that don't know the field are
unaffected.

Also registers PublicDataPlane in the urql cache's keys config (alongside
the existing DataPlane entry) to silence a normalization warning for the
new unkeyed type.

Verified end-to-end against a real local stack: picker lists the seeded
public plane, auto-selects it, and a completed registration produced a
storage_mappings row with the picked plane in data_planes.
- Move PublicDataPlaneNode + a named toPublicDataPlaneNode transform into
  src/api/gql/dataPlanes.ts, alongside the existing DataPlaneNode /
  toDataPlaneNode pair the sibling useDataPlanes() hook uses. The hook
  previously defined its own type and inlined an identity transform,
  diverging from where this codebase already models data-plane nodes.
- Destructure betaOnboard's generateUserClaim args by name instead of
  indexing (args[0], args[1], args[2]) — same positional-args signature
  (shared across all directive types), just clearer at the read site.
- Rename DataPlaneSelector's PREFERRED_DEFAULT to
  DEFAULT_PUBLIC_DATA_PLANE, matching the backend's constant of the same
  meaning and making the value's type (a plane name) obvious from the name.

No behavior change: tsc and lint both pass clean.
The Organization Name field overrides the 2px theme default to
borderRadius: 3 (6px). The picker sat right below it inheriting the
theme default, so the two adjacent inputs had visibly different
corners. Apply the same override.
- Sort options by cloudProvider then region: groupBy only groups
  correctly when the list is ordered by group, which previously worked
  only because plane names embed the provider.
- Extract the region/name label used by both getOptionLabel and
  renderOption so the two can't drift.
- Drop isOptionEqualToValue; value is always drawn from options, so
  MUI's default reference comparison already matches.
- Hoist the input sx to a module const, and reuse hasLength for the
  claim's non-empty check.
CI's Check Quality runs prettier, which I hadn't run locally. Formatting
only, no functional change.
The helper text said 'processed and stored'. That's true for a tenant
left on the trial bucket, but we encourage customers to bring their own
storage, in which case the plane choice doesn't decide where their data
lives. Say only what the choice always controls.
@SeanWhelan
SeanWhelan force-pushed the sean/onboarding-data-plane-picker branch from f3f213e to 47fbe96 Compare September 3, 2026 16:48
@adrian-estuary
adrian-estuary force-pushed the sean/onboarding-data-plane-picker branch from 47fbe96 to fe27956 Compare September 10, 2026 18:56

@adrian-estuary adrian-estuary left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Just a couple of points about not using Zustand and react-intl.

Comment thread src/directives/Onboard/Store/hooks.ts Outdated
Comment on lines +20 to +25
export const useOnboardingStore_requestedDataPlane = () => {
return useLocalZustandStore<
OnboardingState,
OnboardingState['requestedDataPlane']
>(OnboardingStoreNames.GENERAL, (state) => state.requestedDataPlane);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SeanWhelan - I think the plan is to migrate away from Zustand. I would recommend just using react's useState to manage the selected data plane in the onboarding form.

cc: @GregorShear - Correct me if I'm wrong here. 🙏

@GregorShear GregorShear Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally yes, Zustand is overkill in a lot of places and should be removed. It's still appropriate when it shares state for some complex flow (setting up a new capture for example?) or a deep subtree. Not the case here.

(and in some future PR we could remove zustand entirely from Onboarding)

Comment thread src/lang/en-US/Authentication.ts Outdated
Comment on lines +109 to +110
'tenant.dataPlane.label': `Data Plane`,
'tenant.dataPlane.helper': `Where your data is processed. Pick the region closest to your data sources.`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're also migrating away from react-intl. So better to just put the strings in the component.

cc: @GregorShear

{intl.formatMessage({ id: 'tenant.dataPlane.label' })}
</FormLabel>

<Autocomplete

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should add disableClearable to hide the x button to clear the selection. Seems like we don't want this to be unset. Also, the useEffect above will just trigger, setting it back to the default anyway.

Comment on lines +33 to +34
const optionLabel = (option: PublicDataPlaneNode) =>
`${option.region} (${option.name})`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it required to show the full plane name to the user? Seems a bit noisy. Perhaps just the region and suffix would do?

So us-east-1 - c1 (or something like that) versus us-east-1 (ops/dp/public/aws-us-east-1-c1).

Comment on lines +71 to +73
if (error || (!loading && options.length === 0)) {
return null;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should failing API calls log somewhere? What do we use for frontend logging?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh good call

postHog.capture('Entity:Action', {
    someData,
});

);
}

export default DataPlaneSelector;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty minor but would love help establishing a new pattern where we favor named exports

Suggested change
export default DataPlaneSelector;

const optionLabel = (option: PublicDataPlaneNode) =>
`${option.region} (${option.name})`;

function DataPlaneSelector() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
function DataPlaneSelector() {
export function DataPlaneSelector() {

Keep the selected plane in BetaOnboard's local state instead of the
onboarding store, drop react-intl for plain strings, disable clearing,
shorten option labels to region + cluster, log query failures to
PostHog, and switch to a named export.
@SeanWhelan

SeanWhelan commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks both, all addressed in a1cde3e, 90c99d6, eae120d and 4681232:

  • Zustand: removed the store additions. The selected plane is now useState in BetaOnboard, and DataPlaneSelector is a controlled component (value/onChange).
  • react-intl: strings are inline now and the two tenant.dataPlane.* keys are gone.
  • disableClearable: added. Agreed, a plane is always preselected, so clearing it didn't make sense.
  • Label: it's region + cluster now, e.g. us-east-1 c1, under the provider group heading. That still tells apart planes sharing a region, which was the reason for the full name. If a name doesn't parse, it falls back to the full name. The full name is still what gets submitted.
  • Logging: a failed publicDataPlanes query now fires an Onboarding:DataPlanes PostHog event with the error message. The picker still hides itself and signup carries on.
  • Named exports: done.
  • Preselect: no longer hardcodes aws-us-east-1-c1. It picks the newest open plane in AWS us-east-1. publicDataPlanes only returns open planes, so replacing a plane is just opening c2 and closing c1, with no UI PR. This came out of the review on estuary/flow#3318.

I also trimmed the code comments down to the non-obvious ones in line with the feedback on the Bindings PR.

Preselecting a specific plane name meant a UI PR every time a region's
plane was succeeded (c1 -> c2 -> ...). publicDataPlanes only returns open
planes, so defaulting to a region and taking its newest open plane follows
a succession automatically: open the new plane, close the old one.

@adrian-estuary adrian-estuary left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for addressing the feedback! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants