Add bootstrap CI and age adjustment to calculate_c_index - #214
Open
ipezygj wants to merge 1 commit into
Open
Conversation
A C-index of 0.75 from 100 deaths and from 5,000 deaths support very different conclusions, and chronological age alone predicts mortality - so an unadjusted point estimate both hides its uncertainty and rewards a clock merely for correlating with age. - ci_bootstrap_samples=N adds seeded percentile-bootstrap 95% CI columns (subject-level resampling; resamples with zero events are skipped) - adjust_for_age=True computes the C-index on residuals after regressing each predictor on chronological age, mirroring the standardization already used in calculate_mortality_hazard_ratios - defaults unchanged: existing callers get the exact same output - tests: parity with direct lifelines concordance_index, CI brackets and determinism, narrowing with n, and an age-only synthetic clock whose adjusted C-index collapses to 0.5 while a genuinely informative clock retains signal
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two gaps in
calculate_c_index, both acknowledged in the codebase itself: the docstring notes it works "without adjusting for age", and the returned point estimates carry no uncertainty. A C-index of 0.75 from 100 deaths and from 5,000 deaths support very different conclusions — and since chronological age alone predicts mortality, an unadjusted C-index rewards a clock merely for correlating with age (the concern behind #100's age-control question, applied to the C-index path).What
Backwards-compatible — existing callers get the exact same output:
ci_bootstrap_samples=N: seeded percentile-bootstrap 95% CI per clock (CI95_low/CI95_high), subject-level resampling, resamples with zero observed events skipped.adjust_for_age=True: C-index computed on the residuals of each predictor after regressing outage— mirroring the standardizationcalculate_mortality_hazard_ratiosalready applies. The adjusted value answers what does this clock add beyond chronological age.Tests
biolearn/test/test_mortality_c_index.py(4 passing): exact parity with a directlifelines.concordance_indexcall on the default path; CI brackets the estimate and reproduces under the seed; CI narrows with more subjects; and a synthetic age-plus-assay-noise clock whose adjusted C-index collapses to ~0.5 while a genuinely informative clock retains signal beyond age. (One test-design subtlety documented in the file: an exact age copy leaves only float-epsilon regression residuals that are still monotone in age, so the synthetic uses realistic assay noise.)test_model.py's collection error on my machine is a pre-existing missing-local-data issue, verified identical on the untouched base.Context
I work on measurement-integrity tooling for evaluation claims. This is a first small piece of what I'd like to contribute here: reliability-aware reporting (ICC/SEM/minimal-detectable-change utilities, and registering a technical-replicate reference dataset like GSE55763) so that biolearn users can report clock changes against a detection floor. If that direction is welcome, I'll follow up — happy to shape it to the roadmap.