Skip to content

Fix flaky Cypress specs and run the suite once, burning in only changed specs - #1568

Merged
Ekwav merged 11 commits into
mainfrom
cypress-speed-and-flakes
Nov 1, 2026
Merged

Ekwav merged 11 commits into
mainfrom
cypress-speed-and-flakes

Conversation

@Ekwav

@Ekwav Ekwav commented Nov 1, 2026

Copy link
Copy Markdown
Member

CI ran the whole suite three times (about 27 minutes for a passing job), so one flaky spec failed unrelated PRs such as #1567. This fixes the flaky specs at their cause, without retries, and repeats only the specs a change touches.

Flaky specs

  • reduced-motion: this was an app bug. OfflineBanner only listened for offline events, so a connection lost before hydration was never shown. It now also reads navigator.onLine on mount, and the test goes offline before the page loads.
  • search: a flag set by an intercept was checked once; it is now checked with a retrying should.
  • flip-tracking-recovery: failed during the first days of each month, because the calendar shows the month of today − 7 days and today's date wasn't on it. The test now opens the next month in that case.
  • dom-translation: waits for the held history request before checking the translated text.
  • bazaar-snapshot-recovery: rewritten on cy.clock (new cypress/support/clock.ts). The old retry test finished in 0.9 s, so it never actually exercised the 3 s retry; now it does.

Speed: the premium polling and slot-assignment tests run on a faked clock instead of real waits. Three tests were merged into two, with the same coverage; the suite has 280 tests. One pass takes about 4:35 instead of 6:47.

Gate: scripts/verify-cypress.mjs first runs the changed specs, and specs importing a changed support module, two extra times. Then it runs the suite once. The base is the merge commit's first parent for PRs and github.event.before for pushes; manual runs get the full pass only. A passing job is expected to take about 9–13 minutes.

Verified locally: a production test build passed the new gate 3 times in a row (each run: the burn-in specs twice, then 280/280). The five formerly flaky specs passed 10/10 each in fresh processes.

This replaces #1563 and #1564.

🤖 Generated with Claude Code

@Ekwav
Ekwav merged commit 589d52f into main Nov 1, 2026
4 checks passed
@Ekwav
Ekwav deleted the cypress-speed-and-flakes branch November 1, 2026 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant