Skip to content

native: report state of discrete sensors - #374

Open
steigr wants to merge 1 commit into
prometheus-community:masterfrom
steigr:fix-discrete-sensor-state
Open

steigr wants to merge 1 commit into
prometheus-community:masterfrom
steigr:fix-discrete-sensor-state

Conversation

@steigr

@steigr steigr commented Sep 3, 2026 •

Copy link
Copy Markdown

In native mode, discrete sensors (PSU status, CPU_PROCHOT/THERMTRIP, chassis intrusion, ...) are always exported with a state of NaN, and Unknown sensor state is logged on every scrape. go-ipmi's Status() only returns a threshold status for threshold sensors; for discrete ones it returns a hexdump of the state bytes (e.g. 0x0080), which none of the cases matched. This is the Unknown sensor state ... state=0x0080 error from #243.

The state of a discrete sensor is now the worst severity of its asserted events (0 nominal, 1 warning, 2 critical), using the severities go-ipmi already has per sensor type and event/reading type. An unknown event results in NaN.

To check this against FreeIPMI, I compared go-ipmi's severities with FreeIPMI's default sensor state tables for all 440 events of the 76 (event/reading type, sensor type) tables. They match except for 7 events that go-ipmi lacks or rates differently. Those are covered by eventStateOverrides: Physical Security "State Asserted", OS Boot "boot device not specified", System ACPI Power State offset 13 and the reserved Watchdog2 offsets 4-7. With the overrides, all 440 match. The two real differences could also be fixed in go-ipmi instead.

Tested on an AMD X570 board in local mode. PSU1/PSU2 Status, CPU_PROCHOT, CPU_THERMTRIP and ChassisIntr report ipmi_sensor_state 0 with FreeIPMI, NaN with --native-ipmi before this change, and 0 after.

Refs #243

🤖 Generated with Claude Code

@steigr
steigr force-pushed the fix-discrete-sensor-state branch from d568c82 to b44ebfd Compare September 3, 2026 10:38

@SuperQ SuperQ left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix all LLM generated comments to be less crazy.

Comment thread collector_ipmi_native.go Resolved
Comment thread collector_ipmi_native.go Resolved
Comment thread collector_ipmi_native.go Resolved
@steigr
steigr force-pushed the fix-discrete-sensor-state branch from b44ebfd to 2eae54c Compare October 17, 2026 09:04
@steigr steigr changed the title native: fix discrete sensor state reporting as NaN, skip sensors with no valid reading native: report state of discrete sensors Oct 17, 2026
@steigr
steigr force-pushed the fix-discrete-sensor-state branch from 2eae54c to 1e8b14e Compare October 17, 2026 09:14
@steigr
steigr requested a review from SuperQ October 17, 2026 09:27
@bitfehler

Copy link
Copy Markdown
Contributor

Thanks for looking into this, I also already noticed this, but I think this solution might be an oversimplification. If I recall correctly, per the IPMI specification, the state bytes returned have a meaning, but wildly different ones depending on sensor type. FreeIPMI does a lot of work there to neatly press all that into a usable "OK/warning/critical" result. I do have a branch locally where I started work on this, but it's been a while, and I need to re-read the specs to refresh my memory. I will keep this open for now, because the current state is certainly wrong, and I will make a call on this PR once I've gotten back deeper into the subject matter.

@steigr

steigr commented Oct 19, 2026

Copy link
Copy Markdown
Author

have a meaning, but wildly different ones depending on sensor type

I expected this :-) But sounds like FreeIPMI already knows about. I'll check, too.

@steigr
steigr force-pushed the fix-discrete-sensor-state branch from 1e8b14e to 5d36730 Compare October 19, 2026 18:33
@steigr

steigr commented Oct 19, 2026

Copy link
Copy Markdown
Author

@bitfehler, you were right, my first version was too simple.

It now follows FreeIPMI: a discrete sensor is nominal, warning or critical, depending on which event is active.

I compared go-ipmi’s severities with FreeIPMI’s defaults (440 events in 76 tables). They match, except for 7 events. I handle these in a small override table, so the result is the same as FreeIPMI everywhere.

I tested it locally, and it reports the same states as FreeIPMI.

go-ipmi returns a hexdump of the state bytes as the status of a discrete
sensor, which was logged as an unknown state and exported as NaN. Derive the
state from the severity of the asserted events instead, as FreeIPMI does.

Refs prometheus-community#243

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: steigr <me@stei.gr>
@steigr
steigr force-pushed the fix-discrete-sensor-state branch from 5d36730 to 134f24a Compare November 17, 2026 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants