Skip to content

chore: update variance metrics key names and value precision - #393

Merged
sajidkakroo-qntm merged 2 commits into
mainfrom
saj/fix_test_metrics
Oct 7, 2026
Merged

sajidkakroo-qntm merged 2 commits into
mainfrom
saj/fix_test_metrics

Conversation

@sajidkakroo-qntm

@sajidkakroo-qntm sajidkakroo-qntm commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Update:

Here are the examples showing the jump from 15 to 16 decimal digits for RXY_PHI_VARIANCE

  1. 16 digits: https://qa.myqos.com/jobs/a42bbc12-2c71-4ad0-9a3c-7ec7253814b1
["METRICS:FLOAT:runtime:experimental:RXY_PHI_VARIANCE",
2.8198869717398165],
[
"METRICS:FLOAT:runtime:experimental:RXY_PHI_VARIANCE",
2.4674011002723395
 ]
  1. 15 digits (older job from 20th Sep): https://qa.myqos.com/jobs/eb20910f-851d-4714-a23f-fb2ba76ec879
["METRICS:FLOAT:runtime:METRICS:FLOAT:RXY_PHI_VARIANCE",
2.819886971739816],
[
"METRICS:FLOAT:runtime:METRICS:FLOAT:RXY_PHI_VARIANCE",
 2.467401100272339]

def approx(expected: int | float) -> Comparison:
return Comparison(
lambda a: (type(a) is int or type(a) is float) and abs(a - expected) <= 1e-10,
f"≈ {expected} (up to 10 decimal places)",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we should wait for the Selene devs to comment on whether the 1 ULP difference is expected or not. It might be that a certain precision is part of the requirements, or could also come from the LLVM lowering...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, if these are experimental, then it's probably fine to just do an approximate comparison... 🤷🏼‍♂️

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I think let's merge it since it's just for the experimental metrics.
I will keep an eye out and update accordingly based on the response from the Selene devs.

@sajidkakroo-qntm
sajidkakroo-qntm merged commit 0acbcdd into main Oct 7, 2026
5 of 7 checks passed
@sajidkakroo-qntm
sajidkakroo-qntm deleted the saj/fix_test_metrics branch October 7, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants