feat(DENG-11076): add onboarding_completed_by_day_27 and app_version_at_onboarding_completion to retention_clients

[kbammarito]

Description

Add onboarding_completed_by_day_27 and app_version_at_onboarding_completion to retention_clients

• fenix_derived.onboarding_completed_clients_v1 — new table, one partition per day
• fenix.onboarding_completed_clients — new view over it
• retention_clients.view.sql — the shared template gains both columns

All six product views gain them: fenix computes them, the other five render CAST(NULL AS BOOLEAN) and CAST(NULL AS STRING). No existing table changes, and fenix.retention doesn't carry them, so no aggregate grain moves.

app_version_at_onboarding_completion is the version at the completion event, not the metric-date version retention_clients already carries. It is limited to the same day-27 window as the flag, so it reads NULL on exactly the rows the flag does, bar a completion event that carried no version.

Grain, the floor and how to read both columns are in the table's README.md and schema.yaml.

Consumer impact

A new column can break a downstream query two ways: a SELECT * off the view written into a fixed-schema table, or a column landing at the wrong ordinal, since mobile_retention_clients unions the six product views by position and a mismatch there compiles clean and reads wrong. Both columns go last and unconditionally in all six branches, which is what holds the ordinal. Swept across both repos: every live reader names its columns, and nothing reads telemetry.mobile_retention_clients at all. Looker, Redash and ad-hoc queries were not searched.

Follow-ups

• The backfill and the Bigeye monitoring check

Related Tickets & Documents

• DENG-11076

Reviewer, please follow this checklist

[scholtzan]

Integration report

• View full diff, if available (SQL + DAGs) (requires private-bigquery-etl access)
• Private CI run
• Bigeye monitoring dry-run: ✅ plan clean — view plan (requires private-bigquery-etl access)

[kbammarito]

@claude review

[github-actions]

This adds fenix_derived.onboarding_completed_clients_v1 (one row per client per day they fired onboarding.completed, one partition per day), a passthrough view over it, and two columns on the shared retention_clients template — computed for fenix, rendered as typed NULL literals for the other five products so the positional union in telemetry.mobile_retention_clients keeps its ordinals.

Checklist notes: no existing table covers per-client onboarding completion for fenix, so the new dataset isn't duplicating available data; the one in-repo reader of fenix.retention_clients (fenix_derived/new_profile_metrics_marketing_geo_testing_v1/query.sql) names its columns, so no downstream schema needs updating; the table lands in client_id-bearing fenix_derived, which find_glean_targets picks up automatically, so the "no shredder_mitigation label" reasoning in the metadata holds. No backfill.yaml is touched.

The query's tie-breaking is well covered by the two test cases, including the NULL-version and undated-event paths. Three comments inline; the earlier note about IF(cond, TRUE, NULL) making the flag TRUE-or-NULL still stands unaddressed, and the new-column semantics comment below is closely related to it.

[github-actions]

suggestion: client_id and sample_id descriptions are hand-written here and diverge from the canonical text in sql/moz-fx-data-shared-prod/fenix_derived/dataset_schema.yaml, which already defines both. docs/reference/schema_includes.md covers exactly this case, and other tables in this dataset (e.g. fenix_derived/feature_usage_events_v1/schema.yaml) already pull from that file, so these copies will drift the next time the canonical description is edited.

The table-specific caveat can ride along via append:

- name: client_id
  type: STRING
  mode: NULLABLE
  description: !include-field-description
    file: /sql/moz-fx-data-shared-prod/fenix_derived/dataset_schema.yaml
    field: client_id
    append: >-
      Not unique in this table: a client has one row per day on which they fired the event.

Same shape for sample_id. completed_date and app_version are genuinely table-specific and are fine hand-written.

[github-actions]

suggestion: The window this compares against is anchored at the row's metric_date, not at the client's first_seen_date, and first_completed_date is the client's all-time earliest completion. So on the repeat_user / existing_user rows — the majority of retention_clients — the predicate is satisfied by a completion that happened long before the row's 28-day window even opened, and the column degenerates to "ever completed onboarding". Only on rows where metric_date = first_seen_date does it mean what the name says.

Concretely: a client first seen 2025-02-01 who completed onboarding that day gets onboarding_completed_by_day_27 = TRUE on every one of their rows for the next two years, including the row whose day-27 window is 2026-08 — so a cohort cut on this column mixes "completed within 27 days of install" with "completed at some point in the past".

Anchoring on first_seen_date makes the value client-invariant and matches the name on every row:

IF(
  onboarding_completions.first_completed_date
  <= DATE_ADD(clients_daily.first_seen_date, INTERVAL 27 DAY),
  TRUE,
  NULL
)

If the metric-date anchoring is deliberate, the column name and the README.md should say so explicitly ("by day 27 of the retention window", not "by day 27"), because the two readings give different numbers for every non-new-profile row.

[github-actions]

nitpick: This description states the coverage floor as a fact, but the backfill is a follow-up, so once this merges the table holds only forward-fill partitions and the 2025-01-01 floor won't exist for a while. This text is what lands in the BigQuery table description and the data catalog, so a reader has no way to tell. The sibling view metadata gets this right ("Coverage will begin 2025-01-01 when the table is backfilled") — worth matching that wording here, and in the README.md "Floor" section which likewise reads as though the window is already populated.