Skip to content

ci: cache Maven distributions and retry bootstrap downloads - #5422

Open
sunchao wants to merge 1 commit into
apache:mainfrom
sunchao:dev/chao/codex/ci-maven-wrapper-retry
Open

ci: cache Maven distributions and retry bootstrap downloads#5422
sunchao wants to merge 1 commit into
apache:mainfrom
sunchao:dev/chao/codex/ci-maven-wrapper-retry

Conversation

@sunchao

@sunchao sunchao commented Aug 23, 2026

Copy link
Copy Markdown
Member

Why are the changes needed?

A Java CI job can fail before it builds or tests Comet even when its Maven dependency cache is a hit. The cache contains .m2/repository, but the Maven wrapper installs Maven itself under .m2/wrapper/dists. Fresh runners therefore still download the Maven distribution from Maven Central.

For example, the Spark 3.4 / JDK 11 shuffle job on #5403 successfully restored its dependency cache, then received HTTP 429 while downloading apache-maven-3.9.6-bin.zip. No shuffle tests ran. The subsequent diagnostic step also failed because target did not exist, adding a second error that obscured the original setup failure.

This makes a temporary download failure require a manual CI rerun without providing any information about the patch being tested. We should reuse the Maven installation and tolerate brief bootstrap failures while preserving real build and test failures.

Which issue does this PR close? None. This is a CI reliability follow-up motivated by #5403, not a change to that PR's expression implementation.

What changes were proposed in this PR?

The shared Java-test action now treats Maven bootstrap as a separate setup phase. Linux jobs restore a small distribution cache keyed by the wrapper configuration, independently of the dependency cache. The action then runs ./mvnw -B --version with up to four attempts and increasing, jittered delays. Once bootstrap succeeds, it saves the distribution immediately, so a later test failure does not discard a usable Maven installation.

Compilation and tests still use their existing commands exactly once. Maven remains at 3.9.6, suite selection is unchanged, and the existing macOS caching workaround remains in place; macOS receives the bootstrap retry without re-enabling caching. This change is limited to callers of the shared Java-test action, not other Maven callers such as RAT or Spark's own test builders.

Failure reporting also distinguishes missing build output from a failed diagnostic command. The debug listing tolerates an absent root target directory, and artifact uploads tolerate files that were never produced. Existing test reports can now be uploaded after a test failure, while cancellation still skips that upload.

How was this PR tested?

  • Ran the actual bootstrap block with the checked-in Maven wrapper and Maven 3.9.6 on JDK 17 against a localhost HTTP fixture. All seven scenarios passed: cold download, warm reuse, reuse after copying the distribution cache, two HTTP 429 responses followed by success, success on the fourth attempt, persistent HTTP 429 failing after four attempts, and the wrapper's default Java-home-based cache location. Retry sleeps were stubbed and their requested delays checked. The fixture also verified that bootstrap never created build output.
  • Passed 13 grouped offline checks using the actual action blocks: retry counts and jitter bounds, retry exhaustion without an extra sleep, both test commands failing after exactly one install invocation with their exit code preserved, diagnostics with and without target, unchanged existing test/dependency-cache steps, and restore/bootstrap/save ordering.
  • Passed Bash syntax checks for all six shell blocks, actionlint 1.7.12 on the repository workflows and a temporary workflow containing the composite action's actual steps, and git diff --check.

The local cache test copies a real wrapper distribution; it does not emulate the GitHub cache service. Hosted cache integration remains for this PR's CI. Comet's Rust/JVM suites were not rerun locally because the patch changes only the CI action.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant