Skip to content

Fix equivalence harness miscalibration from #777 (DL-16) - #792

Merged
ktehranchi merged 1 commit into
PyPSA:developfrom
ktehranchi:fix/equivalence-harness-bus-allocation
Aug 28, 2026
Merged

Fix equivalence harness miscalibration from #777 (DL-16)#792
ktehranchi merged 1 commit into
PyPSA:developfrom
ktehranchi:fix/equivalence-harness-bus-allocation

Conversation

@ktehranchi

Copy link
Copy Markdown
Collaborator

Changes proposed in this Pull Request

Repairs two defects in the Tier-C equivalence harness that landed on develop with #777 and would make the next run return an untrustworthy verdict. Surfaced by the adversarial review of #778, but not caused by it — both are present on develop today.

Three files: one config line, one waiver, one ledger amendment. No pipeline code changes.

B1 — the usa leg was comparing two different demand pipelines

config.equivalence.yaml:77 pins bus_allocation: breakthrough so the western leg stays apples-to-apples with the anchor, which has no census-population method (git show e7f8bd70:workflow/scripts/build_base_network.py | grep load_weight → empty). config.equivalence-usa.yaml carried no such pin, and build.py copies the candidate's config into the anchor worktree — so both sides ran unpinned at #777's new population default.

Effect on any EQ_INTERCONNECT=usa run: the candidate weights demand by 2020 census county population while the anchor weights by Breakthrough Pd. That diverges Load_t.p_set, Bus.LAF_state, the kmeans geometry (cluster_network.py:68,82 weights on load_weight) and the objective — by the full reallocation. Every finding would read as a migration regression and not one of them would be. DL-13's recorded usa leg stopped reproducing the moment #777 merged.

Fix: add the same pin and comment already present in config.equivalence.yaml:75-77.

B2 — prong 1 reported pass: false for a non-migration reason

load_weight is candidate-only and survives every compared stage: cols2drop in aggregate_to_substations.py:165-188 omits it, cluster_network.py:269 sums it into the clustered frame, and clean_bus_data in add_electricity.py:1058-1067 drops only load_dissag/LAF/LAF_state. So compare.py:174-184 raises a column_set finding at five stages and compare.py:546-549 flips the whole run to failing.

Fix: one waiver, following the Generator/ba_eia precedent at waivers.yaml:59-64.

Why the waiver is sound

Only because of the B1 pin. With breakthrough pinned, load_weight carries the anchor's own legacy Pd weighting, so the column's presence is numerically inert. The two fixes are a package — the waiver alone would suppress a real signal.

Scoping was verified against compare.py's actual is_waived (lines 80-87), not by inspection:

the 5 findings B2 predicts:
  assembled_substation_network     waived=True
  clustered_network                waived=True
  extra_components                 waived=True
  prepared_network                 waived=True
  sectored_network                 waived=True

control - must NOT be over-waived:
  Bus.load_weight  / value        waived=False
  Load.p_set       / value        waived=False
  Bus.v_nom        / column_set   waived=False

load_weight value differences remain live, and demand content stays guarded by the unwaived Load / Load_t.p_set comparisons.

Ledger

Recorded as DL-16 in docs/superpowers/specs/2026-08-07-deltas-ledger.md, marked PENDING COUNTERSIGNATURE per the existing convention.

Checklist

  • I tested my contribution locally and it seems to work fine.
  • Code and workflow changes are sufficiently documented.
  • Changed dependencies are added to envs/environment.yaml.
  • Changes in configuration options are added in all of config.default.yaml.
  • Changes in configuration options are also documented in doc/configtables/*.csv.

Verification performed: waivers.yaml parses (29 entries); the new waiver was exercised against compare.py's verbatim is_waived for both the intended matches and the controls above. No harness run — the Tier-C re-baseline that #778 requires is still outstanding, and this PR does not discharge it. B1 in particular is a config change whose effect can only be confirmed by an actual EQ_INTERCONNECT=usa run.

Two defects landed on develop with PyPSA#777 and would make the next
Tier-C harness run return an untrustworthy verdict.

B1 - the usa leg compared two different demand pipelines.
config.equivalence.yaml:77 pins `bus_allocation: breakthrough` so the
western leg stays apples-to-apples with the anchor, which has no
census-population method. config.equivalence-usa.yaml carried no such
pin, and build.py copies the candidate's config into the anchor worktree,
so BOTH sides ran unpinned at the new `population` default. Candidate
demand was weighted by 2020 census county population, the anchor by
Breakthrough Pd - diverging Load_t.p_set, Bus.LAF_state, the kmeans
geometry (cluster_network.py:68,82 weights on load_weight) and the
objective. Every resulting finding would read as a migration regression.
DL-13's recorded usa leg stopped reproducing.

B2 - prong 1 reported pass: false for a non-migration reason.
load_weight is candidate-only and survives every compared stage, so
compare.py:174-184 raised an unwaived column_set finding at five stages
and compare.py:546-549 flipped the run to failing.

The pin is the substantive fix; the waiver is sound only because of it.
With breakthrough pinned the column carries the anchor's own legacy Pd
weighting, so its presence is numerically inert. Demand content stays
guarded by the UNWAIVED Load / Load_t.p_set comparisons, and load_weight
VALUE findings are deliberately not waived - the waiver matches
kind: column_set only, verified against compare.py's is_waived.

Not yet exercised against a live harness run; the Tier-C re-baseline
PyPSA#778 requires is still outstanding.
@ktehranchi
ktehranchi merged commit 00bbf2b into PyPSA:develop Aug 28, 2026
1 check passed
@ktehranchi
ktehranchi deleted the fix/equivalence-harness-bus-allocation branch August 28, 2026 21:28
ktehranchi added a commit to ktehranchi/pypsa-usa that referenced this pull request Aug 28, 2026
Refreshes the cleanliness sweep onto develop after the pypsa 1.3.0 /
linopy 0.9.1 / pandas 3 migration (PyPSA#778), the config.common.yaml untrack
(PyPSA#791) and the equivalence-harness recalibration (PyPSA#792).

Conflicts, all in favour of develop's migrated code:

- _helpers.py: develop deleted load_network (pypsa.descriptors.Dict and
  override_components are gone in v1); the sweep's own deletions of the
  unreferenced pdbcast / aggregate_* / get_aggregation_strategies /
  load_network_for_plots / setup_custom_logger helpers still apply, so the
  file is a pure deletion relative to develop.
- add_extra_components.py: attach_stores stays deleted (still dead on
  develop -- its only call site is a comment); the sweep's
  copy_timeseries_for_suffix helper is kept.
- summary.py: keeps develop's migrated imports
  (pypsa.statistics.get_bus_and_carrier no longer exists) and drops the
  unreferenced _helpers.configure_logging import along with the __main__
  block that used it; get_energy_total / get_demand_base /
  get_capacity_base / get_capacity_brownfield / get_capital_costs stay
  deleted. _iter_components remains in use by get_energy_timeseries.

No pre-1.3 API is reintroduced and no cleanup hunk was applied on top of
code the migration rewrote. Dependency files are untouched and
workflow/config/ holds only .gitkeep.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant