Skip to content

CLI-mode coverage self-test cannot fail: the fixture supplies the property #77

Description

Problem

testing.yaml gained a sonarqube-cloud-maven-cli leg in #76 to cover coverage_report_paths in CLI analysis mode — the mode that input exists for, since sonar-maven-plugin derives report paths from the project model and the Scanner CLI cannot.

The leg runs the code path but cannot fail if the wiring breaks. lfreleng-actions/test-maven-project declares the property itself:

sonar.coverage.jacoco.xmlReportPaths=core/target/site/jacoco/jacoco.xml,app/target/site/jacoco/jacoco.xml

So the Scanner CLI finds reactor coverage whether or not the lane forwards steps.maven-build.outputs.coverage_report_paths. Delete the wiring and the job still passes with coverage reported.

That property is there on purpose — the fixture's own comment says "Keeping both paths working is the point of this fixture" — so simply removing it would break what the fixture is for.

What a discriminating test needs

maven-build-action emits absolute paths (rooted at $PWD/<path_prefix>), while the fixture declares relative ones. A -Dsonar.coverage.jacoco.xmlReportPaths=... on the command line overrides the properties file, so the two configurations really do differ — the run just has no way to notice.

Options, roughly in order of preference:

  1. A second fixture, or a branch of the existing one, with no sonar.coverage.jacoco.xmlReportPaths. Coverage then appears only if the lane supplies the paths, so deleting the wiring turns the reported figure to zero. Keeps test-maven-project intact for the both-paths-work case it exists to prove.
  2. Assert on the analysis log for the supplied property, which is what issue Wire coverage_report_paths from the build into the Sonar scan #70's Validation section actually asked for. Needs the lane or the scan action to surface the assembled arguments as an output; sonarqube-cloud-scan-action builds them in its Build scanner arguments step but does not expose them.
  3. Assert the resulting coverage figure via the SonarCloud API after the analysis task completes, comparing a reactor-wide value against a single-module one. Heaviest, and couples the self-test to live SonarCloud state.

Wider point

This is the third self-test in the repository that exercises a path it cannot see fail:

Gap Issue Why it cannot fail
Submodule checkout #74 No fixture in the org carries a .gitmodules
Go toolchain selection #75 No fixture carries a toolchain directive
Coverage path forwarding this The fixture supplies the property itself

Each is a case where the failure mode is silence — a green scan that evaluated less, or differently, than intended — which is exactly the docs/BRIEF.md D12 hazard, and exactly what a self-test is supposed to catch. Worth considering whether fixture coverage deserves a deliberate pass rather than three separate patches.

Acceptance

  • A CLI-mode Maven leg fails when coverage_report_paths is not forwarded
  • test-maven-project keeps working for both analysis modes as it does today

Raised from Copilot review feedback on #76, which correctly spotted that the new leg proves less than it appears to.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions