refactor(preingestion): let firmware upload results move their counter - #3743
Merged
Conversation
Let firmware upload Events own the existing Redfish failure records along with the frozen method and outcome counter. Successful routes stay metric-only, failures retain their historical level, message, and context, and the unsupported-multipart sibling preserves the error-then-HttpPush fallback sequence. Paired tests lock every route and outcome. This supports NVIDIA#3734 Signed-off-by: Chet Nichols III <chetn@nvidia.com>
Contributor
Author
|
@coderabbitai full_review, thanks! |
Contributor
WalkthroughPreingestion firmware upload metrics now include BMC addresses and failure details, emit route-specific logs dynamically, distinguish unsupported multipart uploads, and validate counters and historical log output across upload methods. ChangesFirmware upload observability
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
|
✅ Action performedFull review finished. |
wminckler
approved these changes
Jul 20, 2026
10 tasks
chet
added a commit
to chet/bare-metal-manager-core
that referenced
this pull request
Jul 21, 2026
The API and BMC proxy TLS listeners incremented their existing failure counters separately from the ERROR records that explained those failures. Typed sibling `Event`s now pair TCP accept, certificate reload, and handshake failures with their historical records. Counter names, reason labels, messages, peer addresses, and error context remain unchanged; connection attempts and successes stay metric-only rather than adding new log volume. The branch also moves the merged NVIDIA#3743 firmware-upload outcome cases into a table-driven test. Production behavior in that area is unchanged. This supports NVIDIA#3728 Signed-off-by: Chet Nichols III <chetn@nvidia.com>
chet
added a commit
to chet/bare-metal-manager-core
that referenced
this pull request
Jul 21, 2026
The API and BMC proxy TLS listeners incremented their existing failure counters separately from the ERROR records that explained those failures. Typed sibling `Event`s now pair TCP accept, certificate reload, and handshake failures with their historical records. Counter names, reason labels, messages, peer addresses, and error context remain unchanged; connection attempts and successes stay metric-only rather than adding new log volume. The branch also moves the merged NVIDIA#3743 firmware-upload outcome cases into a table-driven test. Production behavior in that area is unchanged. This supports NVIDIA#3728 Signed-off-by: Chet Nichols III <chetn@nvidia.com>
chet
added a commit
to chet/bare-metal-manager-core
that referenced
this pull request
Jul 21, 2026
The API and BMC proxy TLS listeners incremented their existing failure counters separately from the ERROR records that explained those failures. Typed sibling `Event`s now pair TCP accept, certificate reload, and handshake failures with their historical records. Counter names, reason labels, messages, peer addresses, and error context remain unchanged; connection attempts and successes stay metric-only rather than adding new log volume. The branch also moves the merged NVIDIA#3743 firmware-upload outcome cases into a table-driven test. Production behavior in that area is unchanged. This supports NVIDIA#3728 Signed-off-by: Chet Nichols III <chetn@nvidia.com>
chet
added a commit
that referenced
this pull request
Jul 21, 2026
…#3741) The API and BMC proxy TLS listeners incremented their existing failure counters separately from the ERROR records that explained those failures. Typed sibling `Event`s now pair TCP accept, certificate reload, and handshake failures with their historical records. Counter names, reason labels, messages, peer addresses, and error context remain unchanged; connection attempts and successes stay metric-only rather than adding new log volume. The branch also moves the merged #3743 firmware-upload outcome cases into a table-driven test. Production behavior in that area is unchanged. ## Related issues - This supports #3728 - Part of #3169 ## Type of Change - [ ] **Add** - New feature or capability - [ ] **Change** - Changes in existing functionality - [ ] **Fix** - Bug fixes - [ ] **Remove** - Removed features or deprecated functionality - [x] **Internal** - Internal changes (refactoring, tests, docs, etc.) ## Breaking Changes - [ ] **This PR contains breaking changes** ## Testing - [x] Unit tests added/updated - [ ] Integration tests added/updated - [ ] Manual testing performed - [ ] No testing required (docs, internal refactor, etc.) The API and BMC proxy failure matrices use `check_values` to verify that every reason produces one counter increment and one historical ERROR record. Since #3743 merged while this branch was open, it also converts that PR's seven firmware-upload outcome cases to `check_values`; the multipart fallback remains explicit because the ordered pair of attempts is the behavior under test. Verified with: - the focused `carbide-api-core` TLS failure `Event` test - the focused `carbide-bmc-proxy` TLS failure `Event` test - the focused `carbide-preingestion-manager` firmware-upload outcome test - `cargo make format-nightly` - `cargo make clippy` - `cargo make carbide-lints` - `cargo make check-event-names` - `cargo xtask check-metric-docs` ## Additional Notes The firmware-upload changes are test-only; production behavior in that area remains the version merged through #3743. Signed-off-by: Chet Nichols III <chetn@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I was walking the preingestion firmware upload fallback and noticed every terminal result already had both halves of an
Event: a{method,outcome}counter observation, and a nearby record explaining the failure. Multipart being unsupported is the fun edge -- it is a real multipart outcome and then anHttpPushattempt, so flattening the flow would lose useful history.So, this gives the upload outcomes typed
Events that own both signals.FirmwareUploadFinisheddynamically keeps each historical failure level and message, while successful uploads stay metric-only. The unsupported multipart path gets a siblingEventso its fallback-specific WARN keeps the exact schema operators already know.carbide_preingestion_firmware_upload_total, its description, and everymethod/outcomelabel stay put. BMC addresses and error details stay in log-only context. Most importantly, multipart fallback still records two honest observations -- the unsupported multipart result, followed by theHttpPushresult -- because that is what actually happened.And yep, the fallback sequence has a test! I ran:
cargo test -p carbide-preingestion-managercargo make format-nightlycargo make clippycargo make carbide-lintscargo make check-event-namescargo xtask check-metric-docsThe local CodeRabbit and independent Claude passes were clean, too.
This supports #3734
Closes #3734