Skip to content

feat: persist index credentials in the OS keyring (sysand auth login/logout/status/whoami) - #453

Open
consideRatio wants to merge 84 commits into
sensmetry:mainfrom
consideRatio:credential-storage
Open

feat: persist index credentials in the OS keyring (sysand auth login/logout/status/whoami)#453
consideRatio wants to merge 84 commits into
sensmetry:mainfrom
consideRatio:credential-storage

Conversation

@consideRatio

@consideRatio consideRatio commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Implements credential storage and the sysand auth command family, v1 of the design settled in #437 (the reviewed plan was settled in this comment, now a historical snapshot; the maintained version is design/credential-storage.md in this branch, updated as review findings landed).

The v1/whoami endpoint is already deployed on sysand.com (a validating login against it probes v1/whoami).

What this delivers

Store an index credential once with sysand auth login, and sysand reuses it across runs on Windows, macOS, and Linux, with the secret held by the OS keyring, never a plaintext file. Bearer-only v1; SYSAND_CRED_* env vars keep working unchanged and take precedence (CI overrides an interactive login).

Command surface: auth login / logout / status
  • sysand auth login [INDEX_URL] [--token-stdin]
    • No URL resolves the default-index chain (SYSAND_DEFAULT_INDEX, config default = true, else https://sysand.com); more than one configured default errors and asks for an explicit URL. The resolved index is always echoed before any secret entry. There is deliberately no per-subcommand --default-index flag: it would only duplicate the positional argument.
    • Secret via hidden prompt (Enter token for ...:, rpassword) or --token-stdin (trims exactly one trailing newline); never an inline argument. No TTY and no --token-stdin fails fast instead of hanging. Empty token errors.
    • Re-login over an existing entry prints a replacing notice before the write.
    • On a host with no usable keyring backend, login refuses to persist and prints exact SYSAND_CRED_* lines with the secret as a literal <token> placeholder (never echoed; asserted by test).
  • sysand auth logout [INDEX_URL]: removes a stored login (same default-index resolution and echo; logging out of an index with no stored login warns and exits 0, so cleanup is idempotent).
  • sysand auth status: one unified view of everything sysand will authenticate with. Stored entries (key in the exact auth logout <key> form, covered globs, subject and token prefix when a validating login ran, expires in N days / expired, SYSAND_CRED_* shadowing) and env entries, tagged by source; each stored entry shows its persisted validation claim (validated (read/api) or a warn-styled not validated); absence negatives print only when nothing is configured at all; entries covering the resolved default index carry a dim (default index) marker. Never shows secrets. Output is styled with the CLI's existing anstyle tokens and aligned to the same 12-column gutter as the rest of the CLI (plain when piped or under NO_COLOR).
  • sysand auth whoami [INDEX_URL]: query-only live identity check against v1/whoami, using the same credential selection as the runtime (env over stored, and the output names which source was used). Exit 0 only on an accepted credential; rejected/unreachable/rate-limited get distinct nonzero messages. Never writes the store. Errors clearly when the index advertises no API.
  • URL-template index locations (for example a GitLab repository-files-API index, https://gitlab.com/api/v4/projects/<id>/repository/files/{path}/raw?ref=index) are accepted as login/logout/whoami targets: the credential is scoped to the template's literal prefix, and template keys are normalized so login/logout/status round-trip.
Storage: single keyring blob, fail-closed, locked
  • All persisted credentials live in one OS-keyring entry (service sysand, account credentials) holding a versioned JSON blob of records {key, globs, scheme, secret, expires_at?, subject?, token_name?, token_prefix?}.
  • Fail-closed codec: unknown fields round-trip (serde(flatten)), a parse failure surfaces "credential store unreadable" instead of silently clobbering stored credentials. A pre-existing older blob is proven to still parse.
  • Read-modify-write is guarded by a cross-process advisory file lock (via the stable std::fs::File locking API) at a per-user path, bounded wait; never an existence-based lock file.
  • Windows blob cap (~2.5 KB, measured in UTF-16 bytes) yields a clear "store full / token too large" error instead of truncation.
  • Error taxonomy: backend absent (env fallback allowed) vs locked/denied (surfaced with an unlock suggestion and the SYSAND_CRED_* fallback named).
  • Cargo feature layout: record types, codec, and the LockedBlobStore type (generic over a BlobBackend seam) are unconditional core; the OS-keyring backend sits behind a new non-default keyring feature (enabled by the CLI; the js/wasm binding stays green, verified by wasm cargo check). The Linux Secret Service backend is keyring 4.x's pure-Rust zbus, so clippy --all-features works on runners without system libdbus.
Consumption: lazy, no extra requests, env precedence
  • New CredentialStoreAuthentication combinator wraps the existing env policy: runs it first, returns non-4xx untouched, and only on an auth-relevant 4xx reads the credential blob (once per process, cached via OnceCell; spawn_blocking for the sync keyring call) and issues a forced bearer retry when a stored glob matches.
  • Crucially it passes the initial response through: no matching record means the original response is returned with zero extra requests (404 is routine on the resolve path, so a naive retry layer would have doubled round-trips for logged-in users; asserted by request-counting tests).
  • Local/offline commands, public unauthenticated reads, and never-logged-in users never touch the keyring. Steady-state keychain reads are silent per platform.
  • Precedence is SYSAND_CRED_* > stored login > unauthenticated, end to end.
  • After a forced retry from a stored credential still fails with any 4xx and the record's expires_at is past, a hint suggests re-running sysand auth login (any 4xx, since some hosts answer 404 on bad auth); at most once per record per process.
Login validation (always on)
  • Discovery-first: the login fetches sysand-index-config.json once, both for glob scoping and to learn api_root. The API surface is probed only when discovery advertised api_root (now required by the protocol, so api_root is set exactly when an API exists), so static indexes are never phantom-probed.
  • Per-surface probe: unauthenticated baseline, then a forced-auth retry only if the baseline was 4xx. A surface counts as tested only when the credential was actually exercised; a public 200 proves nothing. v1/whoami is forced-only. Probes never follow redirects (a redirected probe counts as not tested, naming the target). A 429 is never a verdict (rate-limited probes count as not tested; this was a spec bug found during implementation and fixed in both code and plan).
  • Refusal rule: store if any exercised surface accepted (warning about the rest); refuse, before any write, only when at least one exercised surface rejected and none accepted; nothing exercised stores as "stored, not validated". Output always scopes the claim: validated (read), validated (api), validated (read, api), or stored, not validated.
  • A rejected login whose read surface challenged with WWW-Authenticate: Basic says the index uses username/password auth and routes to SYSAND_CRED_<X>_BASIC_USER / _BASIC_PASS.
  • A successful whoami probe persists subject, token name/prefix, and expires_at into the record for auth status.
  • There is no validation opt-out, no flag and no core parameter: offline or unreachable indexes degrade gracefully through the refusal rule (nothing exercised the credential, so it stores as "stored, not validated" with warnings), false refusals are engineered out (429 and redirects are never verdicts; acceptance by any exercised surface wins), and SYSAND_CRED_* remains the escape hatch for pathological middleboxes.
  • The login discovery fetch is an unauth baseline with a forced-bearer retry (carrying the just-entered secret) on any 4xx, including 404, distinguishing a hidden discovery document from an absent one. A fully private dynamic index therefore gets its api_root learned, disjoint roots covered, and can validate as validated (read, api); a forced 404 is treated as an authoritative "no document".
Glob scoping
  • Derived globs are globset-escaped (globset::escape(<normalized root>) + **), so URLs containing glob metacharacters, including IPv6 literals like https://[::1]:8000/, cannot silently fail to match themselves (tested).
  • Coverage is anchored on the user-supplied discovery URL and extended with the resolved index_root / api_root only when not already prefix-covered; disjoint hosts get their own escaped glob. Tests assert the discovery document URL, index.json URL, and upload URL each match the derived set.
  • Templated locations, both discovery-advertised index_roots and typed login targets, are anchored at their literal prefix (clamped to at least scheme://authority/; a template with no safe anchor is rejected with a pointer to SYSAND_CRED_*).
  • Stored globs are the login-time snapshot and are never auto-updated from live discovery (the security boundary from the plan's trust model).
Publish integration
  • Bearer selection is now source precedence: env matches first (single match within env; ambiguity errors per source without falling back), then stored logins, replacing the old flat exactly-one-match rule. Trusted publishing is unchanged and still wins in CI.
  • The selected bearer carries provenance, so an upload 401/403 says where the credential came from ("from SYSAND_CRED_<LABEL>" vs "from your stored login for <key>") with matching remediation; a 403 additionally points at sysand auth status (catches a project token scoped to a different project).
  • A stored bearer with a clearly past expires_at (one-hour skew margin) stops publish before the archive upload; the server's 401 remains the authority.
  • Publish reads the keyring only when env has no matching bearer (asserted by panicking-provider tests).
Protocol spec and server counterpart
  • design/index-api-protocol.md gains a Token Identity section specifying GET v1/whoami under the resolved api_root: bearer auth, 200 body (subject.type user/project/oidc with per-type subject.name semantics, token.name null for exchanged OIDC tokens, non-secret token.prefix, expires_at with Z suffix), 401 with unspecified body, MUST NOT consume single-use credentials, MAY rate limit.
  • The endpoint itself is implemented and tested in the sysand-index repo (routed at api/v1/whoami, reusing the existing bearer resolution across all three token types). It is already deployed on sysand.com, satisfying the ordering requirement that whoami be live before this client (a validating login against an official index without it would refuse valid tokens on a 404).
Testing and verification
  • 476 sysand-core tests (with the keyring feature enabled) and 191 sysand CLI tests, all green on the final branch; cargo check -p sysand-core -F networking (keyring off) and wasm cargo check green; prek run -a clean; every commit DCO-signed.
  • CLI integration tests never touch the real OS keyring: a debug-build-only SYSAND_TEST_CREDENTIAL_STORE seam selects a file-backed or absent backend (hard-errors in release builds), enabling end-to-end login/status/logout tests including a PTY-driven hidden-prompt test and a no-secret-leak assertion on the no-keyring path.
  • Notable properties covered by tests: zero keyring reads for public/unauthenticated flows, exactly one blob read per process, original response returned with no extra request on no-match, S1-S4 validation matrix from the plan, refusal never mutates an existing record, IPv6/template glob derivation, pre-B7 blob compatibility.
Deferred and follow-ups

Deferred by design (plan section 10): auth set / auth unset, basic auth via --username, --pattern, and longest-prefix glob tie-breaking (P2, requiring an explicit api_root, already landed in main via #478). Follow-ups noted during implementation: exposing do_auth_* through the py/java bindings, templated login targets beyond the literal-prefix scoping that shipped, and cargo-deny verification of the new deps (keyring, believed MIT/Apache-compatible).

docs.sysand.com documentation. The user-facing docs for this feature are written and reviewed (a multi-lens review pass, fixes applied) on the docs/sysand-auth branch in the sysand-index repo, and are live on the staging docs preview. That branch is 16 pages: the client authentication explanation / how-to / reference, the auth login / logout / status / whoami command pages, publish and configuration cross-links, and the index-side API-token pages. It is deliberately gated on this PR shipping: docs.sysand.com deploys from main, and the docs policy forbids documenting functionality that has not been released, so the branch merges to main (and goes live) only once the release carrying this change is out.


🤖 Generated with Claude Code

https://claude.ai/code/session_01EfyrkJkeo3mRngVoppJjUN

@consideRatio

consideRatio commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator Author

Doing some UX testing, dumping notes here. (Transcripts below refreshed to match the gutter-aligned output that landed after the original capture.)

  • sysand auth status output included no coloring, it should look nicer
  • sysand auth login <gitlab index with {path}> doesn't work yet, but plan was that it should
  • sysand auth whoami felt like a very relevant command that was missing, and there is for example kubectl auth whoami, so adding it makes sense to me
  • Fix output alignment to mirror other commands
  • It is not so obvious that indexes configured are typically not considered, its mostly the default index that is considered with commands. not sure what i suggest if anything, sysand auth status
  • (semi unrelated) sysand publish requires the flag --index which is a bit weird UX wise, while sysand auth login takes an optional positional argument. I think it smells a bit, not sure what i suggest.
  • (unrelated) no nice tab indentation etc on sysand info --iri ... output

Testing with staging.sysand.com

$ sysand auth login https://staging.sysand.com
  Logging in to index `https://staging.sysand.com/`
Enter token for `https://staging.sysand.com/`:
error: credential for `https://staging.sysand.com/` was rejected by the index API (`v1/whoami`) and accepted by no surface; nothing was stored

note: pass `-v`/`--verbose` to output additional logs

$ sysand auth login https://staging.sysand.com
  Logging in to index `https://staging.sysand.com/`
Enter token for `https://staging.sysand.com/`:
      Stored credential for `https://staging.sysand.com/` (validated (api))
      Covers https://staging.sysand.com/**

$ sysand auth login https://staging.sysand.com
  Logging in to index `https://staging.sysand.com/`
Enter token for `https://staging.sysand.com/`:
   Replacing existing credential for `https://staging.sysand.com/`
      Stored credential for `https://staging.sysand.com/` (validated (api))
      Covers https://staging.sysand.com/**

$ sysand auth status
      Stored https://staging.sysand.com/  validated (api)
             patterns: https://staging.sysand.com/**
             subject: user admin
             token prefix: sysand_u_53a8fea5
             expires: 2026-10-17 11:27:09 UTC (expires in 89 days)

$ sysand build
    Building kpar `/home/erik/dev/sensmetry/sysand-test-repo/output/proj0-4.0.21.kpar`
updating file metadata (1/1)
   Including readme from `/home/erik/dev/sensmetry/sysand-test-repo/README.md`
   Including license from `/home/erik/dev/sensmetry/sysand-test-repo/LICENSES/MIT.txt`

$ sysand publish --index https://staging.sysand.com
warning: KPAR does not contain a changelog file CHANGELOG.md;
         it is recommended to provide it to inform users of
         the changes between versions
  Publishing admin/proj0 v4.0.21 to https://staging.sysand.com/
   Published new release successfully

$ sysand auth logout https://staging.sysand.com
 Logging out from index `https://staging.sysand.com/`
     Removed stored credential for `https://staging.sysand.com/`

$ sysand auth status
No credentials configured (no stored logins, no `SYSAND_CRED_*` variables).

Testing with private index on GitHub

$ sysand auth login "https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/"
  Logging in to index `https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/`
Enter token for `https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/`:
      Stored credential for `https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/` (validated (read))
      Covers https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/**

$ sysand info --iri "pkg:sysand/example-publisher-1/example-project-1" --index "https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/"
warning: Direct or transitive usages of SysML v2/KerML standard library packages are
         ignored by default. If you want to process them, pass `--include-std` flag
Name: example-project-1
Publisher: Example Publisher 1
Version: 0.0.1
License: MIT
No usages.

Testing with private index on GitLab

$ sysand auth login "https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index"
  Logging in to index `https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index`
Enter token for `https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index`:
      Stored credential for `https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index` (validated (read))
      Covers https://gitlab.com/api/v4/projects/84113019/repository/files/**

$ sysand auth status
      Stored https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index  validated (read)
             patterns: https://gitlab.com/api/v4/projects/84113019/repository/files/**

$ sysand info --iri "pkg:sysand/example-publisher-1/example-project-1" --index "https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index"
warning: Direct or transitive usages of SysML v2/KerML standard library packages are
         ignored by default. If you want to process them, pass `--include-std` flag
Name: example-project-1
Publisher: Example Publisher 1
Version: 0.0.1
License: MIT
No usages.

@consideRatio
consideRatio force-pushed the credential-storage branch 2 times, most recently from a94ecaf to 99a0bd1 Compare July 20, 2026 11:35
@consideRatio consideRatio changed the title feat: persist index credentials in the OS keyring (sysand auth login/logout/status) feat: persist index credentials in the OS keyring (sysand auth login/logout/status/whoami) Jul 22, 2026
@consideRatio
consideRatio force-pushed the credential-storage branch 2 times, most recently from 18c55d5 to 3dcef4b Compare July 22, 2026 17:18
@consideRatio
consideRatio marked this pull request as ready for review July 22, 2026 17:24
Comment thread design/credential-storage.md Outdated
Comment thread core/Cargo.toml Outdated
Comment thread core/Cargo.toml Outdated
Comment thread design/credential-storage.md Outdated
Comment thread design/credential-storage.md Outdated
Comment thread design/credential-storage.md Outdated
Comment thread design/credential-storage.md Outdated
@consideRatio

consideRatio commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

Ai generated below, will review the response myself now:

EDIT (AI-generated): Followed up on the "non-http(s) should not be accepted anywhere" point. Confirmed a sysand index cannot be reached via file:// or a local path: index locations are http/https only (IndexLocation::parse via validate_url_shape), for both --index and config [[index]] entries. Local paths and file:// URLs are accepted only as project sources (the FileResolver, --from-url / --from-path), never as indexes. So this was intentional design, not accidental file:// index access; the only issue was the imprecise doc wording, now fixed in b495659.

Thanks for the review. I rebased the branch onto main (which now includes #478, the "require explicit api_root" change) and addressed each thread. New commits: 1adaacd (keyring 4.1.5 + std file locking), 73a1b25 (doc clarification), and 6bd7be9 (drop the now-redundant api_root_advertised flag). Details folded below to keep this short.

keyring → 4.1.5  ·  done, but the feature set had to change shape

Bumped to 4.1.5. Heads up: 4.x is a full restructure, it is now a thin facade over keyring-core plus per-backend store crates, and the old flat features (apple-native, windows-native, sync-secret-service, vendored) no longer exist. I settled on:

keyring = { version = "4.1.5", optional = true, default-features = false, features = ["v1"] }

The single v1 feature gives us the 3.x-compatible Entry API (so map_keyring_error and the get/set/delete calls compile unchanged) and pulls exactly the native backends we want, each already target-gated inside keyring's own manifest: Keychain on macOS, Credential Manager on Windows, Secret Service (via zbus + crypto-rust) on glibc Linux.

One deliberate deviation from "Secret Service with vendored libdbus": 4.x's compat layer hardcodes the pure-Rust zbus backend on Linux (the dbus-secret-service/vendored-C path is only wired into keyring's unrelated cli feature). Using zbus is strictly better for us: no system libdbus and no C build at all, so --all-features CI lint on glibc needs no system libraries, and its crypto-rust session keeps openssl out (see next). Behavior is unchanged (still reads/writes the OS Secret Service). The musl exclusion stays as before.

openssl out of the dependency tree  ·  confirmed absent

cargo tree --invert openssl --all-features --target all matches nothing, same for openssl-sys and libdbus-sys. The keyring 4.x zbus backend uses crypto-rust, so nothing pulls openssl in. No feature adjustment was needed.

fd-lock: is it necessary?  ·  no, dropped it for std

Good call. MSRV is 1.97 (well past 1.89 where std::fs::File::lock stabilized), so I removed the fd-lock crate and its feature entry and use the std advisory-locking API directly. One signature note in case it comes up: on this toolchain File::try_lock returns Result<(), std::fs::TryLockError> (with WouldBlock / Error(io::Error) arms), not Result<bool>, so the bounded-wait loop maps WouldBlock to the same poll/LockTimeout and Error to CredentialStoreError::Lock. Exclusive lock preserved; the two lock tests (lock_wait_is_bounded, concurrent_writers_do_not_lose_records) exercise the new path.

Why require api_root to be explicitly advertised?  ·  landed as #478

This is exactly what #478 did (now merged into main): an index has an API iff its discovery document advertises api_root, and sysand publish requires it too, so the asymmetry you flagged is gone. This branch is rebased onto that, and I dropped the api_root_advertised flag it used to carry, ResolvedEndpoints.api_root is now Some exactly when an API is advertised, so the login-probe gate and the whoami gate just check api_root.is_some() (6bd7be9). The design doc (§3 assumption P2, §12) is updated to describe P2 enforcement as landed rather than decoupled/future.

non-http(s) indexes should not be accepted anywhere  ·  agree, but broader than this PR

Agreed in principle. auth login already rejects non-http(s) ("not an HTTP(S) index; nothing to authenticate to"). The wider "index resolution accepts a file:///path elsewhere" behavior is a CLI-wide policy call beyond the credential plan's scope, so I have left it as a noted follow-up rather than folding it into this PR. Happy to open a separate issue if you would like it tracked.

Also applied, no discussion needed:

  • "which URL is glob derivation from?" clarified in 73a1b25: it keys off the login-target index URL (what the user passes to auth login, or the resolved default index), not the discovery document; discovery-advertised index_root/api_root add their own globs per §8.
  • "no way to disable validation" reword and "for an API it does not have": both already in the tree (your committed suggestion 8a6c85c, and the rebase which dropped the stale "runtime plain-URL default" clause from that sentence).

@consideRatio

Copy link
Copy Markdown
Collaborator Author

A five-lens review of this branch (architecture, storage correctness/security, command integration, tests/docs, CLI UX) found no high-severity correctness or security bugs, but produced a set of fixes and simplifications that just landed as 11 new commits. Each is summarized below with an honest assessment, so it is easy to judge per commit whether it earns its place; they are independent enough to drop individually if one looks wrong.

Overall verdict from the review: the scary-looking parts (keyring blob, locking, probe/refusal validation, lazy auth layer) earn their complexity; the removable complexity was in duplicated selection logic, a doubled storage abstraction, and a few message/robustness gaps, which is what these commits address.

f541bf9 test: default the CLI harness to the absent credential store seam

What: sysand_cmd_in_with now sets SYSAND_TEST_CREDENTIAL_STORE=:absent: by default (per-test env still overrides), so no CLI test can fall through to the real OS keyring in debug builds.

Why: cli_publish.rs's basic-auth test reached the stored-credential provider and read the developer's real keyring; on macOS that can pop a keychain prompt, and a genuinely stored credential could alter assertions.

Assessment: clearly good, 6 lines, pure test hygiene closing a real isolation hole. No downside identified.

98c7e39 fix(auth): carve 429 out of forced discovery, humanize refusal wording

What: (1) a 429 discovery baseline no longer triggers the forced credentialed retry (treated like unreachable, matching the probes' existing 429 carve-out); (2) a read-probe 404 refusal hedges with "or no index exists at this URL"; (3) the single-surface refusal message reads "the index rejected the token (v1/whoami answered HTTP 401); nothing was stored" instead of "accepted by no surface" jargon.

Why: the 429 case contradicted the code's own stated rationale (do not spend rate budget or transmit the secret to a throttling host); the 404 case blamed the credential when the URL may simply have no index; the wording was design-doc vocabulary leaking to users.

Assessment: good. (1) closes a genuine design-internal inconsistency that transmitted the secret when policy said not to. (2) and (3) are wording; low risk, +144 test lines pin them.

485e2d8 fix(publish): collapse identical-token bearers, polish credential hints

What: lookup_publish_bearer collapses multi-matches carrying the identical token before erroring AmbiguousPublishBearer; the no-bearer hint interpolates the actual index URL instead of a literal <index-url> placeholder; the expired-credential message drops sub-second precision.

Why: two stored logins whose globs both match the upload URL with the same token worked for reads and whoami but spuriously failed publish; every sibling flow already collapsed identical tokens.

Assessment: good. Fixes a user-visible asymmetry with a concrete failure scenario; hint and timestamp changes are copy polish aligned with stated CLI conventions.

5faeba9 fix(auth): harden credential store locking, redaction, and sync cache

What: lock file moves off session-scoped XDG_RUNTIME_DIR to a stable per-user order (state dir, local data dir, home); list() takes a shared lock instead of exclusive; URL-normalization errors redact embedded userinfo; stored_bearer_map_blocking no longer panics on a cache race; regression tests for Debug redaction and map_keyring_error.

Why: the lock path was env-dependent, so a cron/systemd process and an interactive shell could lock different files while read-modify-writing the same keyring entry, silently losing a record (the branch's most significant correctness finding). The userinfo echo printed passwords into stderr/CI logs in the one place the code knows it handles a credential.

Assessment: clearly good; this is the commit with the highest defect-fixing value on the branch. The lock-path change deviates from the design doc's original wording, which c5d5dd0 updates.

c5d5dd0 docs(design): sync credential-storage.md with the review fixes

What: design doc updated for the 429 carve-out, refusal wording, lock path and shared read lock, covers: label, identical-token collapse, seam default, and a stale CI note (the claim that keyring-gated tests never run in the test lane was wrong given feature unification).

Assessment: good and necessary; the doc and code cross-reference each other, and the review found the doc accurate everywhere else, so keeping it that way is the point. Docs-only, zero runtime risk.

02d8f92 fix(auth): polish CLI UX for login, logout, whoami, and status

What: help-text fixes (double negative removed, dangling cross-reference now points at sysand add --help, status help explains validated (read, api)); whoami does exactly one keychain read instead of two (records passed into core; core can no longer touch a store there by construction); non-default ports included in suggested SYSAND_CRED_* stems; logout-of-nothing points at auth status; remediation hints lead with auth login and mention env vars as the CI path; read-404 hedge on the stored-anyway notice; do_auth_login takes &Runtime for consistency.

Assessment: good, though the largest and most mixed commit (+315/-85 over 5 files). The whoami single-read change is a real behavior fix (two unlock prompts on a locked Linux keyring); the rest is copy and signature polish. The do_auth_whoami signature change (store to records) is the piece worth an extra look.

dcc7d54 fix(auth): idempotent logout, quiet re-login notice, one env classifier

What: logout of a non-existent entry warns and exits 0 (gh/cargo convention) while store failures stay hard errors; the "Replacing existing credential" notice respects --quiet (demoted to the logger, symmetric with "Stored"); the SYSAND_CRED_* suffix classifier is extracted to one shared sysand/src/cred_env.rs module used by all three former copies.

Assessment: good. The exit-code change is the only observable-contract change on the branch (scripts relying on logout-of-nothing failing would break; none plausibly exist). The classifier unification removes a three-way drift hazard for +122 lines of one well-documented module.

ab99440 refactor(auth): share one credential-selection helper across read, whoami, and publish

What: one select_bearer helper (match, identical-token collapse, ambiguity classification) now backs the runtime read layer, whoami, and publish; whoami stops hand-rebuilding a GlobMap; stored_bearer_map_from_records factored out and reused.

Assessment: good but honestly not a size win: net -3 lines excluding its new tests (the review's 100-200 line estimate was high). The value is that collapse/precedence/ambiguity semantics are written once; the previous triplication had already produced one real inconsistency (the publish asymmetry fixed in 485e2d8). One knowing delta: whoami now logs the same invalid-glob warning the runtime path always had (previously silent skip).

b1dea59 refactor(auth): direct store read for publish, 429 carve-out, louder degrade

What: the racy sync cache accessor stored_bearer_map_blocking is deleted; publish does one direct store read via the existing provider seam. The lazy layer and SequenceAuthentication no longer escalate credentials on 429. Read-path store failures log at warn. New CLI test proves a seeded stored credential reaches the upload's Authorization header.

Assessment: mostly good: removes an accessor whose safety rested on a comment about runtime flavor, and the 429 carve-out matches the design's "429 is never a verdict". Two knowing costs: a rare run can now read the store twice (discovery escalation plus publish's direct read; sanctioned by the doc update), and the blanket warn-on-degrade was a UX mistake, corrected in 24f0470.

e51abe4 refactor(auth): collapse the CredentialStore trait into LockedBlobStore

What: the CredentialStore trait (one production implementation) is folded into the concrete LockedBlobStore<B>; the test-only in-memory store becomes an in-memory BlobBackend; whoami's PreloadedCredentials shim becomes CredentialStoreAuthentication::preloaded(records). Default-feature, keyring, and wasm builds all verified.

Assessment: good with a caveat: net only -32 lines (short of the estimated 80-100; cfg reflow and test scaffolding ate the difference), so judge it on shape, not size: one less public abstraction, five signatures lose a generic parameter, and the BlobBackend seam (which does earn its keep for the test seam and musl stub) remains the single extension point. If the py/java bindings ever want their own store type the trait would need reintroducing; today they do not enable keyring at all.

24f0470 fix(auth): keep the absent-keyring read fallback quiet

What: partial correction of b1dea59: BackendAbsent on the read path goes back to debug; denied/unreadable/lock failures keep warning.

Why: on keyring-less hosts (CI, musl, headless) any routine 4xx would print a keyring warning to users who never stored a credential, once per process, with nothing actionable. The accepted blind spot (a broken Secret Service surfaces as "absent" and degrades quietly on the request path) stays diagnosable via auth status and auth login, which report it loudly.

Assessment: good; the warn-everything version would have been CI log noise. Worth knowing the blind spot exists.

Verification: after each integration and at the end: cargo test -p sysand-core -p sysand --all-targets (0 failures), cargo fmt --check clean, cargo clippy --workspace --all-features --all-targets clean. Keyring/Windows/macOS paths compile-checked only; CI should exercise them.

Review items deliberately not acted on (considered and declined): dropping template-URL login targets, the pre-upload expiry fail-fast, and the forward-compat extra-field round-trip; all stay as shipped.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3

@consideRatio

Copy link
Copy Markdown
Collaborator Author

Follow-up: three commits reducing PR size, since the diff was large. Method was top-down: write the inventory of behaviors and prose that SHOULD exist, then cut what falls outside it. Key semantics and secret-safety stay; formatting and restated rationale go. The PR shrinks from 11,639 to 10,373 insertions (roughly 700 of the remainder is Cargo.lock from the keyring dependency tree).

8a7b84c test(auth): keep CLI contracts, drop output-formatting pins (-475 lines, 44 to 28 tests)

What: cli_auth.rs keeps flow contracts (login/status/logout round trip, idempotent logout, re-login replace, whoami identity and source, env-shadows-stored, exit codes per failure class) and all secret-safety tests (PTY no-echo, placeholder-not-secret guidance, no secret on any stream). Dropped: byte-exact golden blocks, 12-column gutter prefixes, NO_COLOR byte-identity, default-index-marker variants (one representative kept), and full-sentence message pins loosened to distinguishing substrings. One golden canary remains for auth status.

Assessment: this is where the fat was. Two judgment calls a reviewer may want back (each one revert away): the template-URL CLI round trip and the read-404-hedge flow test were dropped as variants; both behaviors remain pinned at the core layer.

e2158d8 test(auth): cut formatting pins and merge duplicates, keep the behavior matrix (-442 lines, -10.6%)

What: core test files: 8 tests dropped, 16 merged into 7 table-driven ones, exact-prose assertions loosened to substrings, small mock helpers extracted. Every expect(0)/expect(N) request-count proof and the full validation/locking/selection matrix kept.

Assessment: honest shortfall against a 30-45% target: the core suite turned out to be almost entirely semantic pins, so only 10.6% was genuinely cuttable without deleting behavior coverage. Cutting more would remove matrix cells, which defeats the purpose.

2fd5774 docs(auth): trim rationale comments and the design doc (-375 lines net)

What: narrative 10-25-line doc comments reduced to the load-bearing constraint plus a design-doc section pointer; design/credential-storage.md from 785 to 628 lines (build-phases plan reduced to a landed note, docs checklist to a pointer, duplicated explanations deduplicated); its status banner updated from "plan / draft" to shipped. Deliberately untouched: the lock-path rationale, the why-not-SequenceAuthentication note (defends against a future regression), the fail-quiet/fail-loud store-error taxonomy, and the test-seam safety docs.

Assessment: pure prose, no code semantics changed (checked by build + full suite). The remaining 628 doc lines are normative behavior spec.

Verification after integration: full cargo test -p sysand-core -p sysand --all-targets 0 failures, cargo fmt --check clean, cargo clippy --workspace --all-features --all-targets clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3

Comment thread sysand/Cargo.toml Outdated
Comment thread sysand/src/lib.rs Outdated
Comment thread sysand/src/lib.rs Outdated
Comment thread sysand/src/lib.rs Outdated
Comment thread core/src/commands/publish.rs Outdated
Comment thread core/src/auth.rs Outdated
Comment thread core/src/auth.rs Outdated
Comment thread core/src/auth.rs Outdated
Comment thread core/src/auth.rs Outdated
Comment thread core/src/auth.rs Outdated
…biguous env bearers

Review batch from PR sensmetry#453:

- Convert every long credential/auth user-facing message that was broken
  with string-continuation spaces to break with real newlines instead,
  keeping the existing break points (core/src/auth.rs,
  core/src/commands/auth.rs, sysand/src/lib.rs,
  sysand/src/commands/auth.rs, sysand/src/commands/publish.rs,
  sysand/src/credential_store.rs).
- Replace PublishBearerSource with PublishBearerConflict, carrying the
  conflicting candidates per source: Env names every matching
  SYSAND_CRED_<LABEL> variable, Stored names every matching login key.
  BearerSelection::Ambiguous now carries the full pre-collapse match
  list; the identical-token collapse semantics are unchanged.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Comment thread core/src/commands/auth.rs Outdated
The index.json probes only need the status, so HEAD avoids fetching
index bodies during login validation; whoami and the forced discovery
retry keep GET since their bodies are parsed.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Comment thread sysand/src/commands/auth.rs Outdated
Comment thread sysand/src/commands/auth.rs Outdated
…ove inline tests out

`auth status` used to list `SYSAND_CRED_*` pattern variables leniently
while every credential-using command applied strict validation, so
status could paint a healthy picture of a configuration other commands
reject. The strict validation (label-less names, secrets without a URL
pattern, patterns with no complete scheme, half a basic pair) now lives
in one shared place, cred_env::validated_env_groups, and both the eager
policy build in run_cli and `auth status` go through it with the exact
messages the eager path already used. Stems are checked in sorted order
so the reported error is deterministic.

`auth whoami` keeps its lenient per-group policy on purpose: it must
stay usable against a private index even when an unrelated group is
malformed, and its skip-bad-groups behavior (including invalid glob
patterns) is materially different from the strict validation, so
unifying it would change behavior.

Also move the inline tests module in sysand/src/commands/auth.rs to a
sibling auth_tests.rs wired with the #[path] include idiom, matching
every other file in this branch. Test-support items (core's
credential_store test_support module, the cfg(test) accessor on
StoredBearerAuth) stay in place: test files import them.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Comment thread core/src/commands/auth.rs
…rror

Like the publish conflict change: the error lists every matching
SYSAND_CRED_<LABEL> variable or stored key (first-seen order, deduped)
instead of a bare count, so the user knows exactly what to reconcile.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Comment thread core/src/commands/auth.rs Outdated
The API may answer any error code, so the rejection carries the status
per surface instead of implying a hardcoded 401:
ProbeOutcome.rejected and ValidationRejected.rejected become
Vec<(ProbeSurface, u16)>, the separate read_status field folds away
(the read pair's code drives the 404 hedge), and the stored-anyway
SurfaceRejected notice carries its surface's actual status too. Both
the single-surface and enumerated refusal messages now name each
surface's own answer.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Comment thread core/src/commands/auth.rs Outdated
…ones

The read-surface probe previously detected only a Basic challenge as a
boolean. Now it collects every offered challenge scheme token verbatim
(first-seen order, deduplicated case-insensitively, quote-aware
top-level comma split, auth-param continuations filtered) and carries
the list through ProbeOutcome, ValidationRejected, and the
SurfaceRejected notice. Classification happens at render time via the
shared schemes_include_basic / unsupported_schemes_followup helpers:
Basic keeps the existing SYSAND_CRED_* routing hint byte-identical,
Bearer is the expected scheme, and any other scheme is named verbatim
in a new follow-up line on both the refusal and stored-anyway paths.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Comment thread core/src/commands/auth.rs Outdated
Comment thread core/src/commands/auth.rs Outdated
…r key comment

The unanchorable-template case is rejected on the raw input before the
key is built, so the residual arm is a bug state; and the
normalized-template-key comment now says what differs from Display
(the URL-normalized anchor) instead of gesturing at idempotence.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Comment thread core/src/index_location.rs Outdated
Comment on lines 189 to 210

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid having to do shenanigans like e.g. normalized_template_key, use part of the already-parsed expanded to replace prefix, so that parsing here normalizes the URL, and the Display implementation can then be used as the credential store key without changes.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 1faf45c: IndexUrlTemplate::parse now normalizes the prefix (it parses prefix + a re-encoded probe and strips the probe with strip_suffix, which is collision-proof where searching the full expansion is not), so Display is the canonical text and serves directly as the store key; normalized_template_key and its reassembly are deleted. Suffix stays verbatim (literal post-placeholder text). Pins cover host case, default port, the query-only corner gaining its root /, IDN, probe-aliasing literals, and spelled-vs-canonical expansion equivalence. (Claude, on watch duty for Erik.)

Comment thread core/src/commands/auth.rs Outdated
Comment on lines +424 to +428
let Some((cut, anchor)) = template_anchor_root(prefix) else {
return Err(AuthCommandError::TemplateWithoutAnchor {
url: index_url.to_string(),
});
};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be unreachable, since IndexUrlTemplate::parse tests for exactly this (though check this; if it turns out the check there is ineffective, correct it).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified and done in 1faf45c: with parse-time normalization every template prefix carries an explicit path /, so the anchor is infallible for parsed templates; template_anchor_root now takes &IndexUrlTemplate and returns Url (no Option), and the dead TemplateWithoutAnchor variant and TemplateIndexRootSkipped notice are removed. The https://example.com?f={path} corner (which previously anchored nowhere) now logs in keyed as https://example.com/?f={path} with a host-root glob, tested. (Claude, on watch duty for Erik.)

Comment thread core/src/commands/auth.rs Outdated
expires_at: body.token.expires_at,
}),
Err(err) => {
log::debug!("whoami body was not read: {err}");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate "failed to read" from "failed to parse" and in the latter case include the message that failed to parse (it does not contain secrets).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in bdb12b2: the whoami body handling now distinguishes "was not read" (reqwest error with source chain) from "was not parsed" (serde error plus the offending body text, which is index metadata about the token, never a secret). (Claude, on watch duty for Erik.)

Comment thread core/src/commands/auth.rs Outdated
) -> Option<WhoamiIdentity> {
let parsed = runtime
.block_on(response.bytes())
.map_err(|err| err.to_string())

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This silently discards error source(), which reqwest sometimes populates, and which includes the actual details of the error.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in bdb12b2: probe and discovery errors now flow through format_err, so the reqwest source() chain (where the real detail lives) is preserved; the same fix applied to the other error-to-string sites in the auth commands. (Claude, on watch duty for Erik.)

consideRatio and others added 2 commits July 31, 2026 15:51
reqwest errors flow through format_err (the source chain carries the
actual failure detail), and the whoami body debug log now distinguishes
an unreadable body from an unparsable one, including the offending body
text (index metadata, never a secret).

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
…e key

IndexUrlTemplate::parse now rewrites the literal prefix into the text
url::Url serializes it as (lowercase scheme and host, punycode, default
port dropped, explicit root /), by parsing the prefix with a probe
segment appended and cutting the probe back off. The probe goes on the
prefix alone so suffix text can never move the cut, and the suffix
stays as written.

With Display canonical, the auth credential key for a template is just
IndexLocation Display text: normalized_template_key and its prefix
reassembly are gone. The normalized prefix always carries an explicit
path /, so template_anchor_root is infallible for parsed templates
(every IndexUrlTemplate is built by parse; discovery-advertised roots
included) and now takes the template and returns the anchor directly.
The dead TemplateWithoutAnchor error and TemplateIndexRootSkipped
notice are removed: a placeholder directly after the authority, like
https://example.com?f={path}, now anchors at the host root instead of
being rejected or skipped.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PczC2mqtqFveSLWt9aqof3
Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants