Repository navigation
meta: Running cargo-semver-checks in reverse on std stabilisations #839
Description
It has come up a couple times in the past, including very recently, that something snuck through a PR which resulted in behaviour being stabilised without us intending to do so. It's also almost-happened a few times in the past.
Since it now has support for checking stability attributes, cargo-semver-checks is being added as an experimental job to ensure that we don't break preexisting behaviour on new releases, but a point came up that running it in reverse would tell us exactly what new behaviours are getting stabilised in a given release. Is this something we'd like to approve as an experiment?
cc @obi1kenobi
Activity
obi1kenobi commentedon Jul 26, 2026
I'm happy to support this from the c-s-c end if the team decides to go for it!
Note that c-s-c also has a growing number of allow-by-default lints for additive-only changes with appropriate diagnostic messages. But overall that coverage is much less complete than the breakage-oriented lints.
The "run it in reverse" approach will catch more things but the error messages will be confusing (it'll say "removed" but that actually means things were added, because of being reversed). If that bit of UX friction is acceptable for now, I think this is a sound approach.
joshtriplett commentedon Jul 26, 2026
That's fine for now, though it would be nice in the future to have a mode that intentionally tells us about new additions by design.
obi1kenobi commentedon Jul 26, 2026
Enabling those lints via config would do exactly that, we just need a few hundred more lints to have equivalent coverage. Too few hours in the day! :)
theemathas commentedon Jul 27, 2026
Whose job will it be to check that the changes match the release notes? T-release?
nia-e commentedon Jul 27, 2026
if possible, that would be great, but I don't know their work level and I'm okay saying that release just pings libs to double-check against our FCPs
theemathas commentedon Jul 27, 2026
@rust-lang/release
Mark-Simulacrum commentedon Jul 27, 2026
Ideally, the release notes are produced from libs-api FCPs or similar content and that is checked by stabilization PR reviewer + FCP initiator to be accurate in part with the help of c-s-c.
T-release can help act as a backstop but really shouldn't be the one noticing something has gone wrong, it's been 6+ weeks usually by the time release notes are being produced from the FCP start.
nia-e commentedon Jul 28, 2026
i suppose it depends on how we do it. the example i linked wouldn't have run at all in that idea since it wasn't an FCP; we could run it on every PR, but that requires more integration with rustbot to autoreport the stuff we care about
tgross35 commentedon Jul 28, 2026
(Cc obi1kenobi/cargo-semver-checks#1478)
hanna-kruppe commentedon Jul 28, 2026
Obviously future possibilities that don't exist yet shouldn't block any experimentation with the tools that do exist today, but in the medium-to-long run it seems much, much better to detect unintentional stabilizations (as well as breaking changes) before they are merged. That might mean:
mainbranch has a file listing all the API changes since last beta that c-s-c can detect./x test ... --bless)nia-e commentedon Jul 28, 2026
Per today's libs-api meeting, the vibe was positive but some concerns were raised about noisiness for reviewers. The sketch of what we want would be a freestanding golden file which corresponds to "the output of
c-s-cbetween the last beta branchoff and main", and any actual stable changes would need to touch that (basically hanna's comment above). But yes, +1 to adding this in some capacity ^^