Skip to content

Fix: Derive Gerrit credentials from live inputs - #396

Draft
ModeSevenIndustrialSolutions wants to merge 3 commits into
lfreleng-actions:mainfrom
modeseven-lfreleng-actions:fix/credential-memo
Draft

Fix: Derive Gerrit credentials from live inputs#396
ModeSevenIndustrialSolutions wants to merge 3 commits into
lfreleng-actions:mainfrom
modeseven-lfreleng-actions:fix/credential-memo

Conversation

@ModeSevenIndustrialSolutions

@ModeSevenIndustrialSolutions ModeSevenIndustrialSolutions commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Important

Stacked on #393. This branch descends from fix/open-issues, so its two commits appear here too. Review only the last commit, Fix: Derive Gerrit credentials from live inputs. They will drop out once #393 merges.

Closes #394

The bug

derive_gerrit_credentials was @lru_cache(maxsize=32) keyed on (gerrit_host, organization, gerrit_port), while its answer depended on three things absent from that key:

Read Source In key?
_get_respect_user_ssh_setting() G2G_RESPECT_USER_SSH env var no
get_ssh_user_for_gerrit(...) ~/.ssh/config on disk no
_get_cached_git_user_email() git config user.email no

That flag selects between two entirely different credential sources — the user's own SSH identity, or the {organization}.gh2gerrit service account. So a stale entry does not return a slightly wrong answer; it returns a developer's personal SSH username and email where the service account was required.

Before:

respect=true  -> ('modesevenindustrialsolutions', 'mwatkins@linuxfoundation.org')
respect=false -> ('modesevenindustrialsolutions', 'mwatkins@linuxfoundation.org')
identical: True

After, with no cache clearing between the two calls:

respect=true  -> ('modesevenindustrialsolutions', 'mwatkins@linuxfoundation.org')
respect=false -> ('myorg.gh2gerrit', 'releng+myorg-gh2gerrit@linuxfoundation.org')
identical: False

This was latent, not live. cli._load_effective_inputs sets the flag at line 2330 and the only chain reaching the cache starts at line 2344, so the first read happened after the mutation — verified by instrumenting the call. It sat one new caller or one reordering away from being live, with no error to point at a cache.

The two helpers were lru_cache(maxsize=1) over zero-argument functions, which is a global variable with a confusing spelling rather than a cache: with no key there is one slot, so any second caller collides with the first.

What changed

Nothing in this module is cached any more. The three memos are gone, and so is the SSH-config cache that the first revision of this PR tried to make sound.

_get_cached_git_user_email is now _read_git_user_email, since the old name no longer described it. Every docstring claiming caching was rewritten rather than left to mislead.

Why the SSH-config cache went too

The first revision kept it, keyed on (path, st_dev, st_ino, st_mtime_ns, st_size). Review found two holes, both real:

  • Not thread-safe. The snapshot-then-evict-then-insert sequence lets two callers evict the same key, and the second raises KeyError. Bulk mode runs a ThreadPoolExecutor, and the module docstring advertises thread safety.
  • A stat tuple is not a content fingerprint. An in-place rewrite can preserve inode and size, and coarse-timestamp filesystems or mtime-preserving tools can preserve st_mtime_ns. Adding st_dev/st_ino covered atomic write-and-rename — the easy case — but not the one that matters.

A correct key means hashing the file, which is most of the cost of parsing it, so the cache would pay nearly its own price to avoid itself. That prompted the question I should have asked first — what is it saving?

SSHConfig.load(): median 0.182 ms  max 0.493 ms   (50-host, 1629-byte config)

Once or twice per process, and zero times under GitHub Actions, where respect_user_ssh is false. So it went, and the reasoning is recorded where it used to sit so nobody reinvents it. The module now holds no process-wide state at all, which makes the thread-safety claim in its docstring true by construction rather than by locking correctly.

Removed API

clear_ssh_config_cache() and clear_credential_cache() are deleted, along with the ten setup_method calls to them and the autouse fixture. (An earlier revision of this description promised to keep them; that is no longer accurate.)

These are internal helpers of an application, not library API — __init__.py exports only __version__, there is no __all__, and neither name appears in any documentation. Their only callers were this repository's tests. Keeping them as no-ops would be worse than removing them: a function that answers "cleared" while guaranteeing nothing reproduces, in miniature, exactly the invisible-state problem this PR exists to fix. There is also nothing left to want them for, since every input is now read afresh on each call.

Cost of removing the memos

Measured rather than assumed, since avoiding repeated work was their stated purpose:

after cli.py:2344 after cli.py:767
derive_gerrit_credentials 1 2
get_git_user_email 1 2
SSHConfig.load() 1 2

get_git_user_email() costs ~12 ms (two subprocesses: a git --version validation plus the lookup); SSHConfig.load() ~0.2 ms. In GitHub Actions mode these sources are read zero times.

Net: one extra git config lookup and one extra config parse, ~12 ms, once per process, local mode only. Against _load_effective_inputs at 236 ms and a run that clones over the network, the memos were buying noise at the price of a wrong-identity failure mode.

Validation

Gate Result
pytest 1550 passed, 0 failed, 0 errors, 0 skipped
basedpyright (124 files) 0 errors, 0 warnings
ruff check / format clean, no suppressions added anywhere
aislop ci 100 / 100, 0 findings
prek run --all-files every hook passed

Suite went 1549 → 1553 with the new tests, then → 1550 as three tests were deleted with their subject; delta fully accounted for.

Non-vacuity proven by restoring the decorators — all four new tests fail, the first with exactly the reported symptom:

E AssertionError: assert ('sshuser', 'dev@example.org') == ('testorg.gh2...undation.org')
FAILED ...::test_respect_user_ssh_flag_change_switches_source
FAILED ...::test_git_email_change_changes_derived_email
FAILED ...::test_ssh_config_rewrite_changes_derived_user
FAILED ...::test_ssh_config_appearing_later_is_picked_up

Worth a reviewer's eye

Three existing tests asserted the caching directly and could not survive its removal. Repurposed rather than deleted, each inverted to prove the opposite property — please sanity-check this is the right trade:

  • test_derive_gerrit_credentials_caching..._rereads_its_sources; call_count == 1== 2
  • test_respect_user_ssh_setting_caching..._reads_env_every_call; call_count == 1== 3

Three were deleted outright because their subject no longer exists: test_clear_credential_cache_functionality and the SSH-config polluter/victim pair in test_cache_isolation.py. The pair's property is covered more strongly by test_ssh_config_rewrite_changes_derived_user, which rewrites the same path in place — the pair used two different tmp_path directories, which a path-keyed cache would have separated anyway — with get_ssh_user_for_gerrit left unmocked, so only a genuine re-read yields the new answer.

Also noted, unchanged: derive_gerrit_credentials now runs twice per local run rather than once, from each of its two call sites. Same inputs, same answer — only the work repeats. And TestCaching in tests/unit/test_ssh_config_parser.py is now a misnomer; its docstring is corrected but the class name is left alone to avoid churning node IDs.

Completes issue lfreleng-actions#385. An autouse fixture already cleared
gerrit_urls._BASE_PATH_CACHE; the sibling caches it named were left to
a follow-up.

ssh_config_parser memoises four values, and each derives from state
tests routinely monkeypatch. Two are lru_cache(maxsize=1) over
zero-argument functions reading G2G_RESPECT_USER_SSH and git config
user.email, so with one slot and no key *any* two tests collide and
the second sees the first one's environment. derive_gerrit_credentials
keys on (host, organization, port) while its answer also depends on
those two caches and on ~/.ssh/config, none of which appear in the
key. _ssh_config_cache keys on the config path, never its contents, so
rewriting the file under an unchanged HOME serves the previous parse.

The module already exposed clear_credential_cache(), which resets all
four, but only two test modules called it and only from setup_method:
that guards those files on the way in while leaving the caches
populated for whatever runs next. Calling it from an autouse fixture
covers every test, and clearing on both sides stops a dirty cache
reaching a test as well as leaving one behind. No src/ change was
needed.

No leak between existing tests was demonstrable, so this is latent
rather than live. Every test reaching derive_gerrit_credentials either
patches it wholesale or clears in setup_method, which is why it has
held so far. It would bite whenever someone adds a test that derives
credentials for real, and would surface far from its cause.

The new tests pair a polluter with a victim for each cache. Verified
non-vacuous: with the autouse fixtures disabled all five victims fail,
with stale environment, stale git identity and a stale SSH config
parse respectively.

The explicit clear in test_gerrit_urls_more.py at module scope is now
redundant and goes. The one mid-test survives, because it is not a
boundary: without it the second half of that test never probes at all
and its assertions pass off the value cached by the first half. Its
comment claimed the fixture covered it, which was wrong.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
Closes issue lfreleng-actions#386. Duplicate detection short-circuited on GERRIT_SERVER
and GERRIT_PROJECT from the environment, but by the time it runs those
are usually derived fallbacks rather than operator intent:
_load_effective_inputs derives them and writes them to the environment
first.

Two consequences. The .gitreview resolution below that early return was
effectively dead on the normal path, including the base-ref pinning
added for fork pull requests. And the derived project is a guess taken
from the GitHub repository name, so the OpenDaylight repository
integration-distribution yielded that same string where Gerrit holds
integration/distribution. Detection queried a project that does not
exist and reported no duplicates, silently.

Config values now carry provenance. mark_derived_keys records which
keys hold a derived fallback and is_derived_key reports it, backed by
G2G_DERIVED_KEYS. That is process-scoped configuration, settled once
before the bulk thread pool starts, so unlike per-pull-request state it
is safe to hold in the environment.

apply_parameter_derivation marks only those keys whose environment
value is empty, matching the test apply_config_to_env applies moments
later; a key derivation supplies but the environment already defines
explicitly stays explicit. The save-after-success call site opts out,
since it persists to the config file without ever touching the
environment.

Duplicate detection now short-circuits only when both host and project
are explicit, letting the per-pull-request .gitreview resolve otherwise.
Where .gitreview yields nothing the derived pair still applies, because
returning nothing would disable detection entirely wherever it
previously ran against a derived project; a query against a possibly
wrong project is no worse than no query. The fork-trust handling is
untouched.

Provenance is tracked per field, so a partially configured target keeps
operator intent: where only one of the pair is derived, .gitreview
supplies that field alone and the explicit one survives, and only
where it carries a value. A .gitreview naming a host but no project
parses successfully, and taking its empty project would drop the
project qualifier from the query and search every project on the
server, letting an unrelated change of the same title block a
legitimate submission.

A GERRIT_PROJECT arriving from the configuration file counts as
derived, whichever release or hand wrote it. That file is sectioned
per-organization while a project is per-repository, so a value there is
wrong for every repository in the org but one; treating it as a
fallback keeps it usable while letting .gitreview outrank it, and
covers configurations written before such saves stopped.

Derived parameters persisted to the configuration file now omit
GERRIT_PROJECT. That value is per-repository while the file section is
per-organization, so saving it handed every other repository in the org
one repository's project name, and it returned on the next run as
configuration indistinguishable from operator intent.

Verified against the unmodified detector: the new test for a project
containing a path separator fails there, showing the hyphenated name
in the live query URL.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
Copilot AI balanced review requested due to automatic review settings August 27, 2026 13:27
@github-actions github-actions Bot added the bug Something isn't working label Aug 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes unsound credential memoization so Gerrit credentials reflect live environment, Git identity, and SSH configuration.

Changes:

  • Re-reads credential sources on every derivation.
  • Invalidates parsed SSH configuration using file metadata.
  • Adds regression tests for changing credential inputs.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/github2gerrit/ssh_config_parser.py Updates credential and SSH-config caching.
tests/unit/test_ssh_config_parser.py Adds live-input regression tests.
tests/test_cache_isolation.py Verifies credential state does not leak.
tests/conftest.py Stacked #393 test isolation change.
src/github2gerrit/cli.py Stacked #393 provenance change.
src/github2gerrit/config.py Stacked #393 provenance change.
src/github2gerrit/duplicate_detection.py Stacked #393 duplicate-resolution change.
tests/test_config_helpers.py Stacked #393 tests.
tests/test_duplicate_detection.py Stacked #393 tests.
tests/test_gerrit_urls_more.py Stacked #393 cache-test change.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/github2gerrit/ssh_config_parser.py Outdated
Comment thread src/github2gerrit/ssh_config_parser.py Outdated
Closes issue lfreleng-actions#394. derive_gerrit_credentials memoised on
(gerrit_host, organization, gerrit_port) while its answer depended on
three things absent from that key: the G2G_RESPECT_USER_SSH
environment variable, the SSH config on disk, and the git identity.
Since that flag selects between the user's own credentials and the
organisation service account, a stale entry served personal SSH
credentials where the service account was required.

The failure was latent rather than live: cli._load_effective_inputs
sets the environment variable at line 2330 and the only chain reaching
the cache starts at line 2344, so the first read happened after the
mutation. It sat one new caller or one reordering away from serving
the wrong identity, with no error to point at a cache.

Both helper memos were lru_cache(maxsize=1) over zero-argument
functions, which is a global variable rather than a cache: with no key
there is a single slot, so any second caller collides with the first.
All three memos are gone, and the misleading _get_cached_git_user_email
is now _read_git_user_email.

The parsed SSH config is no longer cached either. Keying on the path
served a stale parse once the file was rewritten, and keying on stat
metadata still collides when an in-place rewrite preserves size and
inode on a filesystem with coarse timestamps; a sound key means
hashing the contents, which costs more than the parse it saves. The
eviction it needed was also not thread-safe, in a module that
advertises thread safety and a tool that runs a thread pool. Parsing
costs around 0.2ms for a fifty-host config, once or twice per process,
so the cache was never worth a correctness risk. Holding no
process-wide state is what makes the thread-safety claim true rather
than aspirational.

Removing the memos costs one extra git config lookup per process,
around 12ms, and only in local mode: GitHub Actions runs never read
these sources at all. The parsed SSH config still loads once. Set
against _load_effective_inputs at 236ms and a run that clones over the
network, the memo was buying noise at the price of a wrong-identity
failure mode.

The autouse fixture clearing these caches around every test goes with
them, along with clear_credential_cache and clear_ssh_config_cache and
the setup_method calls to them: with no state to reset there is
nothing left to clear. Tests that asserted the caching directly now
assert that each source is re-read.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comment thread src/github2gerrit/ssh_config_parser.py

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Credential memo omits its own inputs from the cache key

2 participants