Skip to content

Test and build wheels on Python 3.15 - #770

Open
wbarnha wants to merge 1 commit into
masterfrom
claude/python-3.15-cicd-testing-6jtvbj
Open

Test and build wheels on Python 3.15#770
wbarnha wants to merge 1 commit into
masterfrom
claude/python-3.15-cicd-testing-6jtvbj

Conversation

@wbarnha

@wbarnha wbarnha commented Aug 10, 2026

Copy link
Copy Markdown
Member

Description

Adds Python 3.15 to the test matrix and to the release wheel build.

Important

The 3.15 legs will be red until faust-streaming/mode#87 lands and a mode-streaming release carries it. faust does not import at all on 3.15: typing._eval_type's type_params argument became a required positional, and every Record model resolves its annotations through mode.utils.objects.annotations, so collection dies before the first test runs. This PR is safe to merge before then — the 3.15 rows are advisory and cannot block the merge queue — but it won't go green on its own.

Changes

  • Test matrix — 3.15 in the same shape a stable version gets: Cython on, Cython off, plus a confluent-driver leg.
  • build_wheels — cibuildwheel 4.1.04.2.0, the first release that knows about CPython 3.15. It builds cp315 by default: cibuildwheel gates a Python behind the cpython-prerelease enable group only until its first release candidate, because the ABI is frozen at rc1, so a wheel built against 3.15.0rc1 is forward-compatible with 3.15.0 final. The existing cp31?t-* skip keeps this to cp315 without cp315t, matching the other versions — confirmed with --print-build-identifiers.
  • tox.ini — envlist stopped at 3.12; now mirrors the CI matrix. Separately, its basepython block repeated the whole check-env list on all three lines, so factors like flake8 and typecheck matched every line at once and tox resolved base_python to nothing, leaving those envs on whatever interpreter happened to be running tox. I verified this with tox 4.58 (tox config -e flake8 → empty base_python, falling through to the tox interpreter). They now pin to 3.12, the PYTHON_LATEST the lint job uses.
  • pyproject.toml — added the Programming Language :: Python :: 3.15 classifier (verified valid, so PyPI won't reject an upload).

allow-prereleases: true on setup-python resolves 3.15 to whatever pre-release the runner image publishes — 3.15.0rc1 today — and picks up 3.15.0 final automatically. It leaves the stable rows alone: it widens 3.X to ~3.X.0-0, and a pre-release only wins when no stable release satisfies the spec. Verified against the real actions/python-versions manifest: 3.10 → 3.10.20, 3.12 → 3.12.13, 3.14 → 3.14.7, identical with and without the flag.

The 3.15 rows are advisory (experimental: truecontinue-on-error), so they are excluded from the required check job and cannot stall the merge queue.

Verification

With the mode fix applied locally, against the actual 3.15.0rc1 build the runners use:

result
tests/unit + tests/functional 2207 passed, 4 skipped
tests/integration + meticulous + regression 17 passed, 4 skipped
bandit clean
Cython extensions built
Cold dependency install ~105s (aiohttp/aiokafka build from source — no 3.15 wheels yet), comfortably inside the job's 10-minute cap

Follow-up once mode is released

Bump the mode-streaming floor in requirements/requirements.txt, then move the three 3.15 entries out of include: and into the matrix above to make them required.


Generated by Claude Code

Adds Python 3.15 to the test matrix in the same shape a stable version
gets -- Cython on and off, plus a confluent-driver leg -- and moves the
release wheel build from cibuildwheel 4.1.0 to 4.2.0, the first release
that knows about CPython 3.15.  4.2.0 builds cp315 by default:
cibuildwheel gates a Python behind the `cpython-prerelease` enable group
only until its first release candidate, because the ABI is frozen at rc1,
so a wheel built against 3.15.0rc1 is forward-compatible with 3.15.0
final.  The existing `cp31?t-*` skip keeps this to cp315 without cp315t,
matching the other versions.

`allow-prereleases` on setup-python resolves `3.15` to the newest
pre-release the runner image publishes (3.15.0rc1 today) and will pick up
3.15.0 final with no further change.  It leaves the stable rows alone: it
widens `3.X` to `~3.X.0-0`, and a pre-release only wins when no stable
release satisfies the spec, so 3.10-3.14 still resolve to their newest
stable patch.

The 3.15 rows are advisory (`experimental: true` -> `continue-on-error`)
and will stay red for now.  faust does not import on 3.15 until
mode-streaming ships the fix for `typing._eval_type`, whose `type_params`
argument became a required positional in 3.15: every Record model resolves
its annotations through `mode.utils.objects.annotations`, so collection
dies before the first test runs.  With that fix applied locally the suite
is clean on 3.15.0rc1 -- 2207 passed in unit+functional, 17 in
integration/meticulous/regression, bandit clean, Cython extensions built.
The workflow comment records what to change once a mode-streaming release
carries the fix.

tox.ini's envlist stopped at 3.12; it now mirrors the CI matrix.  Its
basepython block repeated the whole check-env list on all three lines, so
those factors matched every line at once and tox resolved `base_python` to
nothing, leaving flake8/typecheck/bandit and friends on whatever
interpreter happened to be running tox.  They now pin to 3.12, the
PYTHON_LATEST the lint job uses.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jr6Lsfd3fULmP1Gi4b6nDK
@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.05%. Comparing base (e2d801a) to head (d1e6822).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #770      +/-   ##
==========================================
- Coverage   96.07%   96.05%   -0.02%     
==========================================
  Files         103      103              
  Lines       11081    11081              
  Branches     1189     1189              
==========================================
- Hits        10646    10644       -2     
- Misses        343      345       +2     
  Partials       92       92              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

wbarnha commented Aug 10, 2026

Copy link
Copy Markdown
Member Author

CI status

Two red checks, neither of which I can fix from this PR. Details so nobody has to dig through logs.

The three 3.15 legs — blocked on a mode-streaming release

Confirmed as exactly the cause described in the PR body, not something new. From Python 3.15/Cython: false/Driver: aiokafka:

ConftestImportFailure: TypeError: _eval_type() missing 1 required positional argument: 'type_params'
  tests/integration/conftest.py:11: from tests.integration import app as _app_module
  tests/integration/app.py:7:      class Point(faust.Record):
  faust/models/record.py:138:      fields, defaults = annotations(
  .../site-packages/mode/utils/objects.py:469: typ = _eval_type(typ, globalns, localns)

Worth noting the interpreter resolved correctly — pythonLocation: /opt/hostedtoolcache/Python/3.15.0-rc.1/x64 — so the allow-prereleases half of this PR is doing its job. The failure is entirely downstream in mode.

faust-streaming/mode#87 has now merged, but that isn't sufficient on its own: this workflow installs mode-streaming from PyPI, and the newest release there is 0.7.0, published today at 13:49 UTC — before that merge. So it still carries the bug.

This needs a new mode-streaming release cut from mode master. Once one exists:

  1. bump the floor in requirements/requirements.txt,
  2. move the three 3.15 entries out of include: and into the matrix proper to make them required.

Until then the legs stay red. They are advisory (continue-on-error), so they are excluded from the required check job and cannot stall the merge queue — this PR is safe to merge in the meantime.

codecov/project — a mid-run artifact, not a real coverage drop

The reported -63.49% is codecov comparing against a partial run. Its own comment shows the upload count, which has been climbing on each re-fire as jobs land:

re-fire HEAD uploads BASE uploads
16:09 2 31
16:10 3 31
16:10 4 31

Files, Lines and Branches are all identical to base (103 / 11081 / 1189) and codecov/patch is green — nothing in this diff touches coverable code. Once the remaining legs finish uploading it should settle back at ~96%: the 3.15 legs fail before their upload step, so they contribute nothing, and base had no 3.15 legs either — HEAD ends up with the same upload set base had.

I'll confirm that once the run completes and follow up here only if it does not recover.


Generated by Claude Code

wbarnha commented Aug 10, 2026

Copy link
Copy Markdown
Member Author

Run complete: 20 success / 4 failure / 3 skipped (the 3 skips are the release-only wheel/sdist/publish jobs). Follow-up on the two items from my last comment, plus the one new red check.

✅ The required check job passed

Worth stating explicitly, since it's the thing this PR's design hinges on: the three 3.15 legs failed and ✅ Ensure the required checks passing still went green. The continue-on-error wiring keeps a pre-release interpreter out of the required gate exactly as intended, so this PR cannot stall the merge queue.

Python pypy3.11/Cython: false — an unrelated hypothesis timing flake

Not caused by this PR, and not 3.15-related:

tests/meticulous/assignor/test_copartitioned_assignor.py::test_remove_clients
hypothesis.errors.FlakyFailure: ... produces unreliable results:
  Failed on the first call but did not on a subsequent one
Unreliable test timings! On an initial run, this test took 5147.73ms, which
exceeded the deadline of 4000.00ms, but on a subsequent run it took 3519.15ms,
which did not.

hypothesis itself classifies the run as unreliable, and the retry came in ~1.6s under the deadline. PyPy is the slowest leg in the matrix, so it sits closest to that 4s cliff. 1 failed / 2223 passed.

For the record, this PR cannot affect that job: the test-pypy block is byte-identical to master (diff over the job → no change), no requirements/ file is touched, and the diff is 3 files — the workflow's test-pytest matrix, a classifier, and tox.ini. It is also already continue-on-error: true and excluded from check's needs, so it is advisory by existing repo policy.

If it recurs, the targeted fix is @settings(deadline=...) on that test (or deadline=None for the PyPy leg) rather than raising the job timeout — the deadline is per-example, not per-job. Out of scope here; happy to do it separately if you want.

codecov/project — recovered to 96.05%, red on a 2-line delta

As expected it climbed back once all uploads landed: 32.58% → 96.05%, and the "different number of reports" warning is gone. It stays red only because it is now -0.02% (Hits 10646 → 10644) against a zero-tolerance threshold.

The likely cause is the pypy flake above, not this diff: a failed job never reaches its Enforce coverage step, so the PyPy leg uploaded no coverage at all, while it did on the master run being compared against. Two lines that only that leg reaches would produce exactly this. I'm flagging that as the probable explanation rather than a proven one — I haven't confirmed those two specific lines are PyPy-only. What is certain is that this diff contains no Python source changes whatsoever, so it cannot mechanically move coverage, and codecov/patch is green.

Re-running the pypy job should clear both.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants