Add DB size alert threshold with auto-suggestion and archival signal banner

[Ajay-Singh-Adhikari]

Closes #27
Closes #28

What changed

Before this PR, there was no way for an engineer to know whether a client's database needed archival without opening the dashboard, reading raw numbers, and making a manual judgment call. This PR adds a configurable size threshold for the wp_woocommerce_order_itemmeta table, evaluates an archival signal against it on every push, and surfaces the result as a dashboard banner — replacing the manual judgment with a clear automated signal.

Settings: DB Size Alert Threshold

Before: No threshold setting existed. Archival readiness was left entirely to the engineer.

After: A DB Size Alert Threshold field is added under WooCommerce Settings with an MB/GB unit dropdown. The value is always stored in MB internally; the unit dropdown is submitted as a named form field and conversion from GB to MB happens server-side in the sanitize callback — JS is display-only.

When the field is empty, the settings page collects a live metrics snapshot and auto-suggests a threshold using this formula:

avg MB/order        = order_itemmeta_size ÷ total_orders
archivable size     = avg × orders_older_than_retention_window
post-archival (z)   = current_size − archivable_size
suggested threshold = z × 2

A 500 MB minimum floor suppresses the suggestion for sites too small for archival planning — an informational message is shown instead. If no orders are older than the retention window, the suggestion is skipped and the engineer is prompted to set it manually.

Archival signal evaluation

Before: order_itemmeta_size_mb and orders_older_than_threshold were collected but never compared — no signal was evaluated.

After: On every metrics collection, the signal fires when both conditions are true simultaneously:

1. order_itemmeta_size_mb >= alert_threshold_mb
2. orders_older_than_threshold > 0

The second condition prevents false alerts when a large table is caused entirely by recent orders that cannot be archived yet. If no threshold is configured, no evaluation runs.

Result is pushed to BigQuery as:

• woo_archival_signal_active (BOOLEAN)
• woo_alert_threshold_mb (INTEGER)

Dashboard signal banner

Before: No signal was visible on the dashboard. The engineer had to read raw numbers to make a call.

After: A banner appears above the metrics tables — immediately visible without scrolling. Three states:

• Hidden when no threshold is configured
• DB health: OK (green) — conditions are not met
• Archival recommended (red) — both conditions are met

The threshold is read live from the database on every dashboard load (not from the cached snapshot), so the banner reflects the current state immediately when the threshold is changed or cleared.

Files changed

File	Change
class-admin-settings.php	New DB Size Alert Threshold field, auto-suggestion logic, sanitize callback, cache-busting hooks
class-admin-dashboard.php	Archival signal banner rendered above metrics tables
class-data-collector.php	Signal evaluation in collect_woo_order_metrics(), three new fields in format_for_bigquery()
class-bigquery-client.php	Three new schema fields: woo_total_orders, woo_archival_signal_active, woo_alert_threshold_mb
class-live-data.php	Fallback defaults for the two new fields
admin.css	BEM classes for threshold input wrap and signal banner
properf-wordpress-adapter.php	Version bumped: 1.0.3 → 1.1.0

Acceptance criteria

• DB Size Alert Threshold field exists under WooCommerce Settings with MB/GB selector
• Field validates input — rejects zero and negative values, preserves previous value on invalid input
• Auto-suggestion pre-fills threshold from current DB snapshot using the formula when field is empty
• Sites with DB below 500 MB show "too small for archival planning" instead of a low suggested value
• Sites with no archivable orders show "set manually" message
• Signal evaluates: size >= threshold AND orders_older_than_threshold > 0
• If no threshold is configured, no evaluation runs
• woo_total_orders, woo_archival_signal_active, and woo_alert_threshold_mb pushed to BigQuery on every run
• Dashboard banner is visible above metrics tables
• Banner is hidden when no threshold is configured
• Banner reflects the current threshold state immediately — no stale cache

Self-review fixes

A deeper review pass after the initial implementation caught and fixed 11 additional issues:

#	File	Fix
1	class-admin-dashboard.php	(float) cast before number_format — PHP 8.1+ TypeError crash when stale transient has null for order_itemmeta_size_mb
2	class-data-collector.php	Added archival_signal_active: false and alert_threshold_mb: null to no-WC early return — prevented undefined-index notices in format_for_bigquery() on non-WC sites
3	class-admin-settings.php	Changed MB→GB display from Math.round(x*100)/100 to toFixed(4) — prevents lossless round-trip drift (e.g. 1500 MB → 1.46 GB → 1495 MB on save)
4	class-admin-settings.php	Fixed elseif ($suggested > 0) → elseif ($healthy > 0) — wrong branch fired for very small DB sizes where round($healthy * 2) = 0
5	class-data-collector.php	Changed falsy $itemmeta_size ? check to null !== $itemmeta_size — DB permission errors returning null no longer silently set size to 0.0 and permanently suppress the archival signal
6	class-admin-settings.php	Transient write-back after cold collection — Settings page was running full DB queries on every load when transient expired, result was never cached
7	class-admin-dashboard.php	Dashboard now reads archival_signal_active and alert_threshold_mb from the woo metrics snapshot instead of recomputing live — removes duplicate derivation logic and single-sources the signal
8	class-admin-settings.php	Accurate error message on first-ever invalid save — previously said "Previous value restored" even when no prior value existed
9	class-bigquery-client.php	Added ALLOW_FIELD_ADDITION to load job — without it, any existing table with the old schema would fail on the next push
10	class-admin-settings.php	step changed from 0.01 to any for GB mode — browser was rejecting form submission when toFixed(4) produced 4-decimal values like 1.4648
11	class-bigquery-client.php	Marked woo_order_items_size_mb, woo_order_itemmeta_size_mb, woo_orders_older_than_threshold, and woo_query_execution_ms as NULLABLE — all four receive null from the no-WC early return, causing load job failures on non-WC sites

Deploy note

This PR adds three new columns to the BigQuery schema: woo_total_orders, woo_archival_signal_active, and woo_alert_threshold_mb. Any existing BigQuery table receiving pushes will fail on the next run after this PR merges unless these columns are added first. The load job now includes ALLOW_FIELD_ADDITION as a safety net, but the DDL must still run first for tables where these columns don't exist yet.

Ajay is responsible for adding the following fields to the MFM BigQuery table schema before the next scheduled push. In the BigQuery console, open the table → Edit Schema → New fields → Edit as text, paste the JSON below, and save:

[
  {"name": "woo_total_orders", "type": "INTEGER", "mode": "NULLABLE"},
  {"name": "woo_archival_signal_active", "type": "BOOLEAN", "mode": "NULLABLE"},
  {"name": "woo_alert_threshold_mb", "type": "INTEGER", "mode": "NULLABLE"}
]

[Ajay-Singh-Adhikari]

Spec redesigned — QET threshold replaced with order_item_meta size threshold

The original spec proposed a configurable QET alert threshold (ms). After reviewing production data from MFM, that approach was dropped in favour of order_item_meta size + orders older than threshold. Reasons:

Why QET was dropped as the primary signal:

• QET is traffic-dependent — the same DB produces different readings at midnight vs peak hours. The cron fires at a fixed time, so a high reading could mean degradation or just a busy server.
• The baseline locks after the first 10 readings. ProPerf was deployed 17 months after MFM's January 2025 archival, so the baseline was set on a DB already at 12,500 MB — not the clean post-archival state. "2× baseline" fires at 2× an already-elevated starting point.
• We have no reference for when degradation began on MFM. 17.3 GB is "when someone ran an audit," not "when performance started dropping." There's no calibration anchor.

Why order_item_meta size + orders older than threshold:

• Size is traffic-independent — 12,500 MB today is 12,500 MB tomorrow regardless of load
• The two-condition check prevents false alerts from recent order growth where archiving wouldn't help

Signal condition:
order_item_meta size > [configurable size threshold MB] AND orders older than retention window > [configurable minimum count]

How the size threshold is configured:

The settings page shows a breakdown above the threshold field — current table size, total orders, orders older than the retention window, and an estimated post-archival size. The threshold field is pre-filled using that estimate with 50% headroom added.

For an experienced engineer, the breakdown is a verification step. They read the numbers, judge whether the uniform MB/order assumption holds for that client, and override the value if needed — for instance setting a lower threshold for an earlier warning on a high-traffic site.

For an engineer who is not yet familiar with the client, the pre-filled value is the answer. They read the one-line description next to the field, confirm the number looks reasonable, and save. The plugin handles the judgment from that point — the dashboard shows either a green OK badge or an amber "Archival recommended" badge. No need to interpret raw numbers or compare QET values.

[Ajay-Singh-Adhikari]

Formula validation

MFM:

• 12,250 MB current, 411,614 total orders, ~105,766 older than threshold
• Formula gives ~18,400 MB threshold
• Kuldeep confirmed 18–20 GB is MFM's real archival planning signal ✓

Sankalptaru:

• 9.55 MB itemmeta, 7,688 total orders, 2,677 older than 8 years
• Formula gives ~12 MB — correctly suppressed by 500 MB floor
• Message shown: "DB is too small for archival planning"

---

Resetting the DB size alert threshold

The threshold is calculated once from the current DB snapshot and stored as a fixed value. It does not recompute automatically — this is intentional so the reference point doesn't shift after archival.

There is no UI reset. To clear the threshold (e.g. to re-run the suggestion on a fresh DB state after archival), delete the option directly from the database:

DELETE FROM <prefix>_options WHERE option_name = 'properf_order_itemmeta_db_alert_threshold';

Sankalptaru example:

mysql -u root uat_sankalptaru_web -e "DELETE FROM staru_options WHERE option_name = 'properf_order_itemmeta_db_alert_threshold';"

After deleting, reload ProPerf > Settings — the field will be empty and the auto-suggestion will run again from the current snapshot. For sites where the DB is below 500 MB, no suggestion is shown and the threshold must be set manually.

---

[pokhiii]

Substantial feature — closes #27 (archival readiness signal). The mechanics are right: two-condition evaluation, cache-busting on the new setting (add/update/delete), NULLABLE schema fields, sanitize callback with user feedback, 500 MB floor as a thoughtful UX addition. Version bump 1.0.3 → 1.1.0 correct for a new feature.

However, walking issue #27's AC list against the diff surfaces three real gaps and one robustness concern. Details inline; summary of what's covered here:

AC verification against #27

• Field exists in WooCommerce Settings, positive integer, sanitize callback, error on invalid ✓
• 500 MB floor for suggestions ✓ (bonus, not in issue — worth adding to the issue if we want future PRs to preserve it)
• No archivable orders → "set manually" message ✓
• woo_archival_signal_active and woo_alert_threshold_mb pushed to BigQuery ✓
• Dashboard banner with two states, hidden when no threshold ✓
• Banner reflects current threshold state immediately ✓ (via live threshold read + cache-bust hooks)
• total_order_count (woo_total_orders) pushed to BigQuery on every run — AC #1 of #27. Collected in collect_woo_order_metrics() but never added to format_for_bigquery() or the schema. See inline note.
• Settings page shows auto-suggestion with the derivation — #27's UX example is explicit about showing the four numbers (current size, total orders, archivable orders, post-archival estimate) so the engineer can judge whether the uniform MB/order assumption holds. Current notice just says "Pre-filled based on current DB state: estimated post-archival size × 2." See inline note.
• Signal formula divergence — #27 says suggested threshold = post-archival estimate × 1.5 (50% headroom). Code uses × 2. See inline note.

WP best-practices audit

Clean:

• No new SQL — pure feature layered on existing collector paths.
• All new output escaped (esc_html, esc_attr, esc_html_e).
• Capability gate + nonce inherited from Settings API.
• Sanitize callback validates positivity and preserves prior value on invalid input (matches the pattern set on sanitize_threshold_years).
• add_option_ + update_option_ + delete_option_ hooks all wired for the new option — full coverage.
• Schema fields marked NULLABLE on BigQuery side.

Minor:

• sprintf( __( '...%s...%s...' ), ... ) in the banner strings would be more translator-friendly with positional placeholders (%1$s, %2$s). WPCS-level nit only.
• Signal is computed twice — once in collect_woo_order_metrics() (stored in the snapshot, pushed to BQ) and once in render_dashboard() (recomputed live). Since the cache is busted on threshold change, the dashboard could just trust $woo_metrics['archival_signal_active'] and drop the live re-computation. Non-blocking design nit.

Robustness

One inline note on the JS-based GB→MB conversion — a silent misconfiguration path exists if JS is disabled or fails. Details on that inline.

Positive callouts

• Cache-busting on all three lifecycle events (add / update / delete) — better than the pattern used for the earlier settings, and correct.
• 500 MB floor is a good judgment call; the alternative low-value suggestions would have been misleading.
• PR description is genuinely detailed — before/after per file, files-changed table, AC checklist.

Suggested path forward

1. Add woo_total_orders to the schema and push. Small.
2. Expand the suggestion notice to show the derivation (roughly matching #27's UX example).
3. Decide × 1.5 vs × 2 — update either the issue or the code to match.
4. The GB→MB conversion is the only non-issue-AC concern that has a real user impact; see inline for one server-side approach.

Once (1)–(3) are addressed and (4) is either fixed or explicitly deferred, this is approve-ready.

[pokhiii]

All four inline items landed cleanly — see per-thread replies for verification.

What stood out this round is the self-review discipline. Ajay ran a full second pass after addressing my comments and caught 11 additional issues, several of which would have been real bugs in production:

• ALLOW_FIELD_ADDITION on the BigQuery load job (class-bigquery-client.php:156) — this closes an outstanding concern I'd flagged all the way back on PR #20 that had been carried as a manual-runbook step ever since. Now BigQuery auto-adds new columns and the load job survives future schema additions. Big improvement.
• NULLABLE mode on woo_order_items_size_mb, woo_order_itemmeta_size_mb, woo_orders_older_than_threshold, woo_query_execution_ms — all four receive null from the no-WC early return, so without NULLABLE the load job would have failed on any non-WC site. Would only have surfaced when someone deployed this to a WC-less install.
• (float) cast before number_format on the dashboard banner — prevents PHP 8.1 TypeError when a stale transient has null for order_itemmeta_size_mb.
• elseif ( $healthy > 0 ) correction — was $suggested > 0, which routed very small DBs (where round($healthy * 2) = 0) to the wrong branch. Same fix also came up on thread 2.
• Dashboard reads archival_signal_active from the snapshot instead of recomputing live — closes the "duplicate derivation logic" design nit I flagged in the summary body last time.
• toFixed(4) for MB→GB display + step="any" for GB mode — prevents lossy round-trips like 1500 MB → 1.46 GB → 1495 MB on save. Cheap fix, real bug for anyone editing an existing threshold.
• null !== $itemmeta_size guard instead of a falsy check — DB permission errors returning null no longer silently coerce to 0.0 and permanently suppress the archival signal.

That's the kind of pass I'd want done on every non-trivial PR — take a step back after addressing feedback and reread the whole change with fresh eyes. Genuinely good engineering discipline.

Approving.