Skip to content

Add X chr functionality#66

Draft
AprilYUZhang wants to merge 43 commits into
AlphaGenes:develfrom
AprilYUZhang:X_chr
Draft

Add X chr functionality#66
AprilYUZhang wants to merge 43 commits into
AlphaGenes:develfrom
AprilYUZhang:X_chr

Conversation

@AprilYUZhang

@AprilYUZhang AprilYUZhang commented May 9, 2026

Copy link
Copy Markdown

Issue #47

Comment thread src/alphaimpute2/Imputation/ImputationIndividual.py Outdated
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py Outdated
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py Outdated
Comment thread src/alphaimpute2/Imputation/Heuristic_Peeling.py
pointEstimates[1, i] *= e
pointEstimates[2, i] *= 1 - e
pointEstimates[3, i] *= 1 - e
elif isXChrom:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am struggling since I am doing this on the fly and can’t check. @XingerTang can you have a look too?

Also, we don’t need the isXChrom block for the second haplotype below?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"ind.haplotypes[0][i] != 9" means this individual is female. If this individual is female, the segregation probability must be pointEstimates[0, i] *= e; pointEstimates[1, i] *= e; pointEstimates[2, i] *= 1 - e; pointEstimates[3, i] *= 1 - e. Don't need to change if sirehap0 != 9 and sirehap1 != 9 and sirehap0 != sirehap1: because it is not possible for isXChrom. If you think if isXChrom: is easy to understand, I can change elif to if.

Comment thread src/alphaimpute2/alphaimpute2.py
Comment thread src/alphaimpute2/alphaimpute2.py Outdated
Comment thread src/alphaimpute2/alphaimpute2.py Outdated
Comment thread src/alphaimpute2/alphaimpute2.py Outdated

@gregorgorjanc gregorgorjanc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AprilYUZhang great that you are pushing this functionality! I reviewed on the go without access to computer to look into the code, so I wasn’t sure in some places if the code is correct or not or it’s just my lack of deep understanding. @XingerTang your review would be also appreciated.

@gregorgorjanc gregorgorjanc changed the title update X chr Add X chr functionality May 10, 2026
@gregorgorjanc

Copy link
Copy Markdown
Member

@AprilYUZhang I left some more comments at de2b6c8#r184944694 on your latest commit

@gregorgorjanc

Copy link
Copy Markdown
Member

Github is behaving odd lately - you might need to toggle file by clicking the > symbol to hide and the unhide file contents to see the comments

@XingerTang XingerTang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Pre-commit not passed
  • pytest failed on my end:
(phase) xtang3@S37-50MAQ05D AlphaImpute2 % pytest tests/functional_tests 
======================================== test session starts =========================================
platform darwin -- Python 3.11.11, pytest-9.0.1, pluggy-1.6.0
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/xtang3/AlphaImpute2
configfile: pyproject.toml
plugins: benchmark-5.2.3
collected 2 items                                                                                    

tests/functional_tests/run_func_test.py F.                                                     [100%]

============================================== FAILURES ==============================================
_______________________________________________ test_1 _______________________________________________

    def test_1():
        """basic functionality test with pedigree only mode"""
        os.system(
            "AlphaImpute2 -genotypes tests/functional_tests/test_1/genotypes.txt -pedigree tests/functional_tests/test_1/pedigree.txt -ped_only -phase_output -seg_output -out tests/functional_tests/outputs/test_1"
        )
        assert os.path.exists("tests/functional_tests/outputs/test_1.genotypes")
        assert os.path.exists("tests/functional_tests/outputs/test_1.haplotypes")
        assert os.path.exists("tests/functional_tests/outputs/test_1.segregation")
    
        genotypes = read_file("tests/functional_tests/outputs/test_1.genotypes")
        expected_genotypes = read_file("tests/functional_tests/test_1/true_genotypes.txt")
>       assert genotypes == expected_genotypes
E       AssertionError: assert [['1', 1.0, 2....0, 0.0, 0.0]] == [['1', 1.0, 2....0, 0.0, 0.0]]
E         
E         At index 2 diff: ['3', 1.0, 2.0, 0.0, 0.0, 1.0] != ['3', 0.0, 2.0, 0.0, 0.0, 1.0]
E         Use -v to get more diff

tests/functional_tests/run_func_test.py:53: AssertionError
---------------------------------------- Captured stdout call ----------------------------------------
------------------------------------------
               AlphaImpute2               
------------------------------------------
Version: 0.0.3
Email:   alphagenes.dev@gmail.com
Website: https://github.com/AlphaGenes
------------------------------------------
Reading in AlphaGenes format: tests/functional_tests/test_1/genotypes.txt
Read in: 0.01 seconds

         Pedigree Imputation Only         
------------------------------------------
Number of peeling cycles: 4
Final cutoff: 0.1

Imputation cycle 1
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
Peel down: 4.86 seconds
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
Peel up: 4.89 seconds

Imputation cycle 2
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
Peel down: 0.00 seconds
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
Peel up: 0.00 seconds

Imputation cycle 3
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
Peel down: 0.00 seconds
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
Peel up: 0.00 seconds

Imputation cycle 4
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
Peel down: 0.00 seconds
False -1
[[2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]
 [2.8239352e-01 3.0398920e-01 3.3998200e-01 3.9997000e-01 4.9994999e-01]
 [2.1760647e-01 1.9601080e-01 1.6001801e-01 1.0003000e-01 4.9999999e-05]]
False 1
[[0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]
 [0.25 0.25 0.25 0.25 0.25]]
Peel up: 0.00 seconds
Core peeling cycles: 9.75 seconds
Heuristic Peeling: 11.28 seconds

           Writing Out Results            
------------------------------------------
Write out: 0.00 seconds
Full Program Run: 11.52 seconds

Comment thread src/alphaimpute2/alphaimpute2.py Outdated
Comment thread src/alphaimpute2/alphaimpute2.py Outdated
Comment thread src/alphaimpute2/alphaimpute2.py Outdated
Comment thread src/alphaimpute2/alphaimpute2.py Outdated
@XingerTang XingerTang marked this pull request as draft May 12, 2026 09:04
@XingerTang

Copy link
Copy Markdown
Contributor

No tests are written for this code. Convert this PR to a draft.

Comment thread src/alphaimpute2/tinyhouse
@AprilYUZhang

AprilYUZhang commented Jun 5, 2026

Copy link
Copy Markdown
Author

Here I calculated per-locus, per-individual Pearson correlation between true and estimated segregation states.
For each individual i and each locus j:
cor(true[i, j], est[i, j])
Then I summarised two ways by individual, by locus
Both split by male / female and by generation.

======================================================================
  Matrix shape: 1000 individuals x 2000 loci
  Males: 495  Females: 505
======================================================================

  ALL INDIVIDUALS
  by-ind mean (across loci)            mean=0.8803  var=0.0299  min=-0.0042  max=1.0000
  by-ind var  (across loci)            mean=0.0278  var=0.0040  min=0.0000  max=0.3562

  ALL LOCI
  by-loci mean (across inds)           mean=0.8803  var=0.0001  min=0.8553  max=0.8962
  by-loci var  (across inds)           mean=0.0575  var=0.0001  min=0.0433  max=0.0825

  MALE — by individual
  by-ind mean (across loci)            mean=0.9054  var=0.0254  min=0.5626  max=1.0000
  by-ind var  (across loci)            mean=0.0199  var=0.0028  min=0.0000  max=0.3562

  FEMALE — by individual
  by-ind mean (across loci)            mean=0.8556  var=0.0330  min=-0.0042  max=1.0000
  by-ind var  (across loci)            mean=0.0354  var=0.0050  min=0.0000  max=0.3528

  MALE — by locus
  by-loci mean (across male inds)      mean=0.9054  var=0.0001  min=0.8788  max=0.9217
  by-loci var  (across male inds)      mean=0.0453  var=0.0001  min=0.0268  max=0.0759

  FEMALE — by locus
  by-loci mean (across female inds)    mean=0.8556  var=0.0001  min=0.8164  max=0.8822
  by-loci var  (across female inds)    mean=0.0683  var=0.0001  min=0.0444  max=0.1032

======================================================================
image
PER-GENERATION 1:
======================================================================
  Matrix shape: 200 individuals x 2000 loci
  Males: 100  Females: 100
======================================================================

  ALL INDIVIDUALS
  by-ind mean (across loci)            mean=1.0000  var=0.0000  min=1.0000  max=1.0000
  by-ind var  (across loci)            mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  ALL LOCI
  by-loci mean (across inds)           mean=1.0000  var=0.0000  min=1.0000  max=1.0000
  by-loci var  (across inds)           mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  MALE — by individual
  by-ind mean (across loci)            mean=1.0000  var=0.0000  min=1.0000  max=1.0000
  by-ind var  (across loci)            mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  FEMALE — by individual
  by-ind mean (across loci)            mean=1.0000  var=0.0000  min=1.0000  max=1.0000
  by-ind var  (across loci)            mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  MALE — by locus
  by-loci mean (across male inds)      mean=1.0000  var=0.0000  min=1.0000  max=1.0000
  by-loci var  (across male inds)      mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  FEMALE — by locus
  by-loci mean (across female inds)    mean=1.0000  var=0.0000  min=1.0000  max=1.0000
  by-loci var  (across female inds)    mean=0.0000  var=0.0000  min=0.0000  max=0.0000

======================================================================
  PER-GENERATION 2:
======================================================================
  Matrix shape: 200 individuals x 2000 loci
  Males: 82  Females: 118
======================================================================

  ALL INDIVIDUALS
  by-ind mean (across loci)            mean=0.5774  var=0.0000  min=0.5774  max=0.5774
  by-ind var  (across loci)            mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  ALL LOCI
  by-loci mean (across inds)           mean=0.5774  var=0.0000  min=0.5774  max=0.5774
  by-loci var  (across inds)           mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  MALE — by individual
  by-ind mean (across loci)            mean=0.5774  var=0.0000  min=0.5774  max=0.5774
  by-ind var  (across loci)            mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  FEMALE — by individual
  by-ind mean (across loci)            mean=0.5774  var=0.0000  min=0.5774  max=0.5774
  by-ind var  (across loci)            mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  MALE — by locus
  by-loci mean (across male inds)      mean=0.5774  var=0.0000  min=0.5774  max=0.5774
  by-loci var  (across male inds)      mean=0.0000  var=0.0000  min=0.0000  max=0.0000

  FEMALE — by locus
  by-loci mean (across female inds)    mean=0.5774  var=0.0000  min=0.5774  max=0.5774
  by-loci var  (across female inds)    mean=0.0000  var=0.0000  min=0.0000  max=0.0000

======================================================================
  PER-GENERATION 3:
======================================================================
  Matrix shape: 200 individuals x 2000 loci
  Males: 104  Females: 96
======================================================================

  ALL INDIVIDUALS
  by-ind mean (across loci)            mean=0.9775  var=0.0029  min=0.6082  max=1.0000
  by-ind var  (across loci)            mean=0.0179  var=0.0019  min=0.0000  max=0.3294

  ALL LOCI
  by-loci mean (across inds)           mean=0.9775  var=0.0002  min=0.9329  max=0.9989
  by-loci var  (across inds)           mean=0.0206  var=0.0002  min=0.0001  max=0.0644

  MALE — by individual
  by-ind mean (across loci)            mean=0.9731  var=0.0046  min=0.6082  max=1.0000
  by-ind var  (across loci)            mean=0.0218  var=0.0028  min=0.0000  max=0.3294

  FEMALE — by individual
  by-ind mean (across loci)            mean=0.9824  var=0.0010  min=0.8261  max=1.0000
  by-ind var  (across loci)            mean=0.0137  var=0.0009  min=0.0000  max=0.1852

  MALE — by locus
  by-loci mean (across male inds)      mean=0.9731  var=0.0004  min=0.9066  max=0.9999
  by-loci var  (across male inds)      mean=0.0260  var=0.0004  min=0.0000  max=0.0876

  FEMALE — by locus
  by-loci mean (across female inds)    mean=0.9824  var=0.0001  min=0.9460  max=0.9995
  by-loci var  (across female inds)    mean=0.0146  var=0.0002  min=0.0000  max=0.0634

======================================================================
  PER-GENERATION 4:
======================================================================
  Matrix shape: 200 individuals x 2000 loci
  Males: 107  Females: 93
======================================================================

  ALL INDIVIDUALS
  by-ind mean (across loci)            mean=0.9648  var=0.0077  min=-0.0042  max=1.0000
  by-ind var  (across loci)            mean=0.0269  var=0.0025  min=0.0000  max=0.3259

  ALL LOCI
  by-loci mean (across inds)           mean=0.9648  var=0.0002  min=0.9162  max=0.9921
  by-loci var  (across inds)           mean=0.0344  var=0.0002  min=0.0077  max=0.0807

  MALE — by individual
  by-ind mean (across loci)            mean=0.9883  var=0.0006  min=0.8488  max=1.0000
  by-ind var  (across loci)            mean=0.0091  var=0.0005  min=0.0000  max=0.1507

  FEMALE — by individual
  by-ind mean (across loci)            mean=0.9379  var=0.0146  min=-0.0042  max=1.0000
  by-ind var  (across loci)            mean=0.0474  var=0.0040  min=0.0000  max=0.3259

  MALE — by locus
  by-loci mean (across male inds)      mean=0.9883  var=0.0001  min=0.9527  max=0.9999
  by-loci var  (across male inds)      mean=0.0095  var=0.0002  min=0.0000  max=0.0518

  FEMALE — by locus
  by-loci mean (across female inds)    mean=0.9379  var=0.0006  min=0.8383  max=0.9867
  by-loci var  (across female inds)    mean=0.0614  var=0.0007  min=0.0158  max=0.1535

======================================================================
  PER-GENERATION 5:
======================================================================
  Matrix shape: 200 individuals x 2000 loci
  Males: 102  Females: 98
======================================================================

  ALL INDIVIDUALS
  by-ind mean (across loci)            mean=0.8816  var=0.0159  min=0.3414  max=1.0000
  by-ind var  (across loci)            mean=0.0940  var=0.0094  min=0.0000  max=0.3562

  ALL LOCI
  by-loci mean (across inds)           mean=0.8816  var=0.0011  min=0.7648  max=0.9504
  by-loci var  (across inds)           mean=0.1088  var=0.0012  min=0.0415  max=0.2122

  MALE — by individual
  by-ind mean (across loci)            mean=0.9204  var=0.0106  min=0.5626  max=1.0000
  by-ind var  (across loci)            mean=0.0651  var=0.0075  min=0.0000  max=0.3562

  FEMALE — by individual
  by-ind mean (across loci)            mean=0.8412  var=0.0183  min=0.3414  max=1.0000
  by-ind var  (across loci)            mean=0.1240  var=0.0096  min=0.0000  max=0.3528

  MALE — by locus
  by-loci mean (across male inds)      mean=0.9204  var=0.0013  min=0.8079  max=0.9881
  by-loci var  (across male inds)      mean=0.0743  var=0.0018  min=0.0027  max=0.2038

  FEMALE — by locus
  by-loci mean (across female inds)    mean=0.8412  var=0.0028  min=0.6577  max=0.9515
  by-loci var  (across female inds)    mean=0.1395  var=0.0021  min=0.0299  max=0.2715

======================================================================

@XingerTang

Copy link
Copy Markdown
Contributor

@AprilYUZhang Very nice plots! Could you try to implement this: #66 (comment)

As that one is segregation-related.

@AprilYUZhang

Copy link
Copy Markdown
Author

@AprilYUZhang Very nice plots! Could you try to implement this: #66 (comment)

As that one is segregation-related.

see 648b0cc

@gregorgorjanc

Copy link
Copy Markdown
Member

I am slowly catching up. One thing I noticed in the summary from @XingerTang are high uncalled rates with X chr, say Uncalled rate: 0.495, so stats in #66 (comment) for X chr and A chr are not comparable

@gregorgorjanc

Copy link
Copy Markdown
Member

@AprilYUZhang that accuracy plot is very effective!!!!! How did you produce it? @XingerTang would that be a useful standard summary for the development!?

Looking at the stats I can see clear expected difference, higher correlation in males, so that is good to see. Generation 2 is odd though, but I think you are looking into that.

can I check what you do with any potential 9s in the output!? Hmm, what are you actually correlating here - allele dosage or called genotypes or genotype probabilities? All are valid targets, wit allele dosages being the most straightforward as there are no 9s in there.

@XingerTang

Copy link
Copy Markdown
Contributor

@AprilYUZhang Thank you for your update. It is a surprise to see that the accuracy is not changed by your updated code. Maybe the same limits have already been applied in other places in the code.

Nevertheless, I now find that I cannot convince myself of the reason why the accuracy of segregation of females should be lower than that of males. Both females and males only have two options out of four. They should have the same base probability (a random choice should yield an accuracy of 0.5). For the X chromosome, the uncertainty always comes from the mother's side. And both males and females have a mother. So I would expect no difference in the accuracy of segregation.

I tried different examples to see if the segregation is correctly inferred, and I think everything looks as expected.

Later, I noticed that the correlation favours the evaluation of the segregation probabilities of the autosomal chromosome,

For true = [0, 0, 1, 0]:

  • an example more likely to appear in an autosomal case: [0, 0.2, 0.7, 0.1], the correlation is 0.96490128
  • an example that's likely to appear in the sex chromosome case: [0, 0, 0.7, 0.3], the correlation is 0.90453403

This is not the sole example that "similar" segregation probabilities have higher correlation accuracy in the autosomal case. I suspect that the fact that the sex chromosome has only two options actually makes it less likely to achieve higher correlation accuracy than the autosomal chromosome.

The later the generation, the more confident we are with the inheritance pattern, so a lot of segregation probabilities for X chromosomes are [0, 0, 0.99, 0.01] or even [0, 0, 1, 0]. But if there is a mistake in the inheritance pattern, then the accuracy would be very close to 0, while the spread-out autosomal segregation probabilities may lead to better results in such a case.

I think what you did earlier for the Argmax match accuracy is a more reliable indicator of the actual accuracy. But if we want to do the comparison, then we may need to apply the same to the autosomal chromosomes. Is it convenient for you to do so?

@XingerTang

Copy link
Copy Markdown
Contributor

I am slowly catching up. One thing I noticed in the summary from @XingerTang are high uncalled rates with X chr, say Uncalled rate: 0.495, so stats in #66 (comment) for X chr and A chr are not comparable

@gregorgorjanc The uncalled rate is not actually the uncalled rate; it is the proportion of males who always have their paternal haplotype written as 9s. But they are essentially not phased, so it is also fair to exclude them from the switch error rate calculation.

@XingerTang

Copy link
Copy Markdown
Contributor

@AprilYUZhang that accuracy plot is very effective!!!!! How did you produce it? @XingerTang would that be a useful standard summary for the development!?

Looking at the stats I can see clear expected difference, higher correlation in males, so that is good to see. Generation 2 is odd though, but I think you are looking into that.

can I check what you do with any potential 9s in the output!? Hmm, what are you actually correlating here - allele dosage or called genotypes or genotype probabilities? All are valid targets, wit allele dosages being the most straightforward as there are no 9s in there.

These plots are nice and informative, maybe a better way to display than the tables. But the correlation is not the best metric here (my above comment explains a little about this). Here we are checking the correlation accuracy of segregation probabilities. So far, I think 9s are mostly from the haplotypes of the male individual (placeholders only). Shouldn't be a worry for us.

@XingerTang

Copy link
Copy Markdown
Contributor

@AprilYUZhang, could you also add a format checker for the pedigree file? I input a pedigree file with no sex specified with -x_chr flag on, but the program runs until it encounters the first place where the sex is needed for some probability calculation, and raises an error that is complaining that there is no sex specified for the individuals. The error message might be too vague for a regular user to understand what is happening.

@XingerTang XingerTang force-pushed the devel branch 2 times, most recently from b945593 to 0301d3a Compare June 12, 2026 20:15
@XingerTang

XingerTang commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

I updated the accuracy test report table; it should now look like the following:

=============================================================  Accuracy =============================================================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff_norm ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------------------------------------------------------- genotypes -------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
pop_only   0.1456
ped_only   0.2919
combined   0.0892
pop_only_x 0.1119
ped_only_x 0.3120
combined_x 0.1275
------------------------------------------------------------ haplotypes -------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
pop_only   0.6551
ped_only   0.4773
combined   0.3095
pop_only_x 0.0603
ped_only_x 0.0501
combined_x 0.0311
------------------------------------------------------------ segregation ------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
ped_only   0.2160
combined   0.2149
ped_only_x 0.2792
combined_x 0.3044
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ind_corr ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------------------------------------------------------- genotypes -------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
pop_only   0.9810
ped_only   0.8970
combined   0.9930
pop_only_x 0.9890
ped_only_x 0.8940
combined_x 0.9860
------------------------------------------------------------ haplotypes -------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
pop_only   0.6680
ped_only   0.8470
combined   0.9260
pop_only_x 0.7850
ped_only_x 0.8660
combined_x 0.9430
------------------------------------------------------------ segregation ------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
ped_only   0.9300
combined   0.9310
ped_only_x 0.8720
combined_x 0.8670
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ marker_corr ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------------------------------------------------------- genotypes -------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
pop_only   0.9530
ped_only   0.7700
combined   0.9800
pop_only_x 0.9730
ped_only_x 0.7920
combined_x 0.9710
------------------------------------------------------------ haplotypes -------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
pop_only   0.4730
ped_only   0.6860
combined   0.8810
pop_only_x 0.9970
ped_only_x 0.9980
combined_x 0.9990
------------------------------------------------------------ segregation ------------------------------------------------------------
Method     Value     
-------------------------------------------------------------------------------------------------------------------------------------
ped_only   0.9680
combined   0.9690
ped_only_x 0.9470
combined_x 0.9370

Only the population values are preserved. For segregation probs, we only count the individuals that have meaningful values, that is, from generation 3 to generation 5.

For the new metric diff_norm, it is the norm of the difference between true and expected, divided by the norm of the true. The values for haplotypes for the X chromosome are really small, it is because of the 9s of males.


bits can be improved:

  • correlation values are rounded to 3 decimal places, but 4 are displayed
  • can add a line of description for each of the metrics
    • diff_norm is the lower the better, while the correlation is larger the better
  • x_chr haplotype excluding the placeholder haplotypes of the males
  • concatenated method name


def generateGenoProbs():
global geno_probs
error = 0.0001

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactor the code so that the value assignment follows the order of 4x4x4


def generateGenoProbs_Xchr_male():
global geno_probs_Xchr_male
error = 0.0001

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactor the code to follow the order as the autosomal

(10, 10, 10, 4), 0.25, dtype=np.float32
) # Because 9 indexing for missing.
generateGenoProbs()
geno_probs_Xchr_male = np.full((10, 10, 10, 4), 0.25, dtype=np.float32)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We start with 0.25 and then in the next function, we change some of these but not all of them, do we ever wrongly use the unmodified 0.25s.

@XingerTang

Copy link
Copy Markdown
Contributor

Updated accuracy report table:

  • Fixed previously mentioned issues
  • Added a new metric correct_rate for segregation probability
  • Added visualised bars
==============================================================  Accuracy ==============================================================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ correct_rate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Summing up the probabilities of the true state from the output data divided by the number of loci being counted.
------------------------------------------------------------- segregation -------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.953 | ############################################################################################.. |
combined             0.953 | ############################################################################################## |
ped_only_x_chr       0.942 | ########################################...................................................... |
combined_x_chr       0.933 | .............................................................................................. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff_norm ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The metric diff_norm is the norm of the difference between the true and output divided by the norm of the true data. The lower the value, the better the accuracy.
-------------------------------------------------------------- genotypes --------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.146 | #######################....................................................................... |
ped_only             0.292 | #####################################################################################......... |
combined             0.089 | .............................................................................................. |
pop_only_x_chr       0.112 | #########..................................................................................... |
ped_only_x_chr       0.312 | ############################################################################################## |
combined_x_chr       0.128 | ################.............................................................................. |
------------------------------------------------------------- haplotypes --------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.655 | ############################################################################################## |
ped_only             0.477 | ##################################################............................................ |
combined             0.310 | ########...................................................................................... |
pop_only_x_chr       0.527 | ##############################################################................................ |
ped_only_x_chr       0.439 | ########################################...................................................... |
combined_x_chr       0.275 | .............................................................................................. |
------------------------------------------------------------- segregation -------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.216 | #............................................................................................. |
combined             0.215 | .............................................................................................. |
ped_only_x_chr       0.279 | ###################################################################........................... |
combined_x_chr       0.304 | ############################################################################################## |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ind_corr ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Pearson correlation evaluated at individuals, ranged from 0 to 1.
-------------------------------------------------------------- genotypes --------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.981 | ##################################################################################............ |
ped_only             0.897 | ##............................................................................................ |
combined             0.993 | ############################################################################################## |
pop_only_x_chr       0.989 | ##########################################################################################.... |
ped_only_x_chr       0.894 | .............................................................................................. |
combined_x_chr       0.986 | #######################################################################################....... |
------------------------------------------------------------- haplotypes --------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.668 | .............................................................................................. |
ped_only             0.847 | #############################################################................................. |
combined             0.926 | ########################################################################################...... |
pop_only_x_chr       0.785 | ########################################...................................................... |
ped_only_x_chr       0.866 | ###################################################################........................... |
combined_x_chr       0.942 | ############################################################################################## |
------------------------------------------------------------- segregation -------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.930 | ############################################################################################.. |
combined             0.931 | ############################################################################################## |
ped_only_x_chr       0.872 | #######....................................................................................... |
combined_x_chr       0.867 | .............................................................................................. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ marker_corr ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Pearson correlation evaluated at markers, ranged from 0 to 1.
-------------------------------------------------------------- genotypes --------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.953 | #################################################################################............. |
ped_only             0.770 | .............................................................................................. |
combined             0.980 | #############################################################################################. |
pop_only_x_chr       0.973 | ##########################################################################################.... |
ped_only_x_chr       0.792 | #########..................................................................................... |
combined_x_chr       0.971 | #########################################################################################..... |
------------------------------------------------------------- haplotypes --------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.473 | .............................................................................................. |
ped_only             0.686 | ######################################........................................................ |
combined             0.881 | ########################################################################...................... |
pop_only_x_chr       0.997 | #############################################################################################. |
ped_only_x_chr       0.998 | #############################################################################################. |
combined_x_chr       0.999 | ############################################################################################## |
------------------------------------------------------------- segregation -------------------------------------------------------------
Method               Value     
---------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.968 | ###########################################################################################... |
combined             0.969 | ############################################################################################## |
ped_only_x_chr       0.947 | #############################................................................................. |
combined_x_chr       0.937 | .............................................................................................. |


@XingerTang

Copy link
Copy Markdown
Contributor

Replaced the norm_diff by abs_diff as norms are not appropriate for our case:

=============================================================================================================================  Accuracy ==============================================================================================================================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ abs_diff ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The metric abs_diff is the sum of absolute difference divided by the norm of the number of loci being counted. 
The lower the value, the better the accuracy.
----------------------------------------------------------------------------------------------------------------------------- genotypes ------------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.023 | #############################.......................................................................................................................................................... |
ped_only             0.096 | ####################################################################################################################################################################################### |
combined             0.009 | ....................................................................................................................................................................................... |
pop_only_x_chr       0.009 | ....................................................................................................................................................................................... |
ped_only_x_chr       0.072 | #####################################################################################################################################.................................................. |
combined_x_chr       0.012 | ######................................................................................................................................................................................. |
----------------------------------------------------------------------------------------------------------------------------- haplotypes -----------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.303 | ####################################################################################################################################################################################### |
ped_only             0.161 | ####################################################################################................................................................................................... |
combined             0.068 | ###################.................................................................................................................................................................... |
pop_only_x_chr       0.147 | ##########################################################################............................................................................................................. |
ped_only_x_chr       0.102 | ###########################################............................................................................................................................................ |
combined_x_chr       0.040 | ....................................................................................................................................................................................... |
---------------------------------------------------------------------------------------------------------------------------- segregation -----------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.094 | ###.................................................................................................................................................................................... |
combined             0.094 | ....................................................................................................................................................................................... |
ped_only_x_chr       0.116 | #######################################################################################################................................................................................ |
combined_x_chr       0.134 | ####################################################################################################################################################################################### |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ correct_rate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Summing up the probabilities of the true state from the output data divided by the number of loci being counted.
---------------------------------------------------------------------------------------------------------------------------- segregation -----------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.953 | ###################################################################################################################################################################################.... |
combined             0.953 | ####################################################################################################################################################################################### |
ped_only_x_chr       0.942 | ###############################################################################........................................................................................................ |
combined_x_chr       0.933 | ....................................................................................................................................................................................... |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ind_corr ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Pearson correlation evaluated at individuals, ranged from 0 to 1.
----------------------------------------------------------------------------------------------------------------------------- genotypes ------------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.981 | ################################################################################################################################################################....................... |
ped_only             0.897 | #####.................................................................................................................................................................................. |
combined             0.993 | ####################################################################################################################################################################################### |
pop_only_x_chr       0.989 | ###############################################################################################################################################################################........ |
ped_only_x_chr       0.894 | ....................................................................................................................................................................................... |
combined_x_chr       0.986 | ##########################################################################################################################################################################............. |
----------------------------------------------------------------------------------------------------------------------------- haplotypes -----------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.668 | ....................................................................................................................................................................................... |
ped_only             0.847 | #######################################################################################################################................................................................ |
combined             0.926 | ############################################################################################################################################################################........... |
pop_only_x_chr       0.785 | ##############################################################################......................................................................................................... |
ped_only_x_chr       0.866 | ####################################################################################################################################................................................... |
combined_x_chr       0.942 | ####################################################################################################################################################################################### |
---------------------------------------------------------------------------------------------------------------------------- segregation -----------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.930 | ####################################################################################################################################################################################... |
combined             0.931 | ####################################################################################################################################################################################### |
ped_only_x_chr       0.872 | ##############......................................................................................................................................................................... |
combined_x_chr       0.867 | ....................................................................................................................................................................................... |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ marker_corr ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Pearson correlation evaluated at markers, ranged from 0 to 1.
----------------------------------------------------------------------------------------------------------------------------- genotypes ------------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.953 | ###############################################################################################################################################################........................ |
ped_only             0.770 | ....................................................................................................................................................................................... |
combined             0.980 | ####################################################################################################################################################################################### |
pop_only_x_chr       0.973 | ################################################################################################################################################################################....... |
ped_only_x_chr       0.792 | ###################.................................................................................................................................................................... |
combined_x_chr       0.971 | ###############################################################################################################################################################################........ |
----------------------------------------------------------------------------------------------------------------------------- haplotypes -----------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pop_only             0.473 | ....................................................................................................................................................................................... |
ped_only             0.686 | ##########################################################################............................................................................................................. |
combined             0.881 | #############################################################################################################################################.......................................... |
pop_only_x_chr       0.997 | ######################################################################################################################################################################################. |
ped_only_x_chr       0.998 | ######################################################################################################################################################################################. |
combined_x_chr       0.999 | ####################################################################################################################################################################################### |
---------------------------------------------------------------------------------------------------------------------------- segregation -----------------------------------------------------------------------------------------------------------------------------
Method               Value     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ped_only             0.968 | #################################################################################################################################################################################...... |
combined             0.969 | ####################################################################################################################################################################################### |
ped_only_x_chr       0.947 | #########################################################.............................................................................................................................. |
combined_x_chr       0.937 | ....................................................................................................................................................................................... |

@XingerTang XingerTang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the population imputation code and left some comments.

ind.current_haplotypes[0][:] = pat_hap
ind.current_haplotypes[1][:] = mat_hap
if ind.isXChr and ind.sex == 0:
ind.current_haplotypes[0][:] = 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the values won't be used, but is there a reason why we assign 0 instead of 9 here?

Comment on lines 524 to +545
@@ -529,10 +539,17 @@ def haplib_sample_alt(sample, bw_library, ind, random_samples):
tmp_mat_prob = hap_lib_prop
scale = rec_rate * rec_rate

geno_probs[i, 0] = tmp_pat_prob[0] * tmp_mat_prob[0] * scale
geno_probs[i, 1] = tmp_pat_prob[0] * tmp_mat_prob[1] * scale
geno_probs[i, 2] = tmp_pat_prob[1] * tmp_mat_prob[0] * scale
geno_probs[i, 3] = tmp_pat_prob[1] * tmp_mat_prob[1] * scale
if isXChr and sex == 0:
# Force pat=0: only j=0 and j=1 are possible
geno_probs[i, 0] = tmp_mat_prob[0] * scale # pat fixed to 0
geno_probs[i, 1] = tmp_mat_prob[1] * scale # pat fixed to 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For X chromosomes in males, there could possibly be only one recombination on its maternal haplotype.

It shouldn't be in this loop of for i in range(4), which iterates over four possible recombination states. And the value of scale multiplied by the tmp_mat_probcan only take values from either 1 - rec_rate or rec_rate.

Similarly, for the X chromosome of female individuals, its paternal haplotype cannot be recombined, so there should only be two cases to iterate through as well.

elif count1 > 0:
mat_hap[i] = 1

pat_hap = np.full(nLoci, 0, dtype=np.int8)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the other comment, I understand the values won't be used, but is there a reason why we assign 0 instead of 9 here?

mat_hap[i] = 1

pat_hap = np.full(nLoci, 0, dtype=np.int8)
return pat_hap, mat_hap

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The get_consensus for the autosomal chromosome assigns the genotype value as well.

Comment on lines +1065 to +1078
for i in range(nLoci):
count0 = 0
count1 = 0
for j in range(nHaps):
val = samples[j].haplotypes[1][i]
if val == 0:
count0 += 1
elif val == 1:
count1 += 1

if count0 > count1:
mat_hap[i] = 0
elif count1 > 0:
mat_hap[i] = 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original count_regional_rec returns rec_scores that are based on the region of the target marker, and later, the majority vote is based on the comparison against the values of rec_scores. But the calculation of rec_scores is absent here.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds X-chromosome (“-x_chr”) support across core imputation/phasing/peeling logic, along with new functional + accuracy tests and a substantial documentation refresh to describe usage and file formats.

Changes:

  • Add sex-chromosome-specific handling for males (hemizygous) across phasing consensus, BW reference-library construction, genotype/phase alignment, and heuristic peeling segregation math.
  • Add new functional test fixtures for sex-chromosome scenarios and extend accuracy tests to run X-chromosome simulations.
  • Improve CLI/docs/examples (new Sphinx docs, version flag handling, example scripts) and expand CI Python-version matrix.

Reviewed changes

Copilot reviewed 49 out of 54 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
tests/functional_tests/test_sex/true-sex.with_recom.haplotypes Adds expected haplotype truth set for X-chr functional scenario (with recombination).
tests/functional_tests/test_sex/true-sex.with_recom.genotypes Adds expected genotype truth set for X-chr functional scenario (with recombination).
tests/functional_tests/test_sex/true-sex.with_recom_missing.haplotypes Adds expected haplotype truth set for X-chr functional scenario (with recombination + missingness).
tests/functional_tests/test_sex/true-sex.with_recom_missing.genotypes Adds expected genotype truth set for X-chr functional scenario (with recombination + missingness).
tests/functional_tests/test_sex/true-sex.no_recom.haplotypes Adds expected haplotype truth set for X-chr functional scenario (no recombination).
tests/functional_tests/test_sex/true-sex.no_recom.genotypes Adds expected genotype truth set for X-chr functional scenario (no recombination).
tests/functional_tests/test_sex/true-sex.no_recom_missing.haplotypes Adds expected haplotype truth set for X-chr functional scenario (no recombination + missingness).
tests/functional_tests/test_sex/true-sex.no_recom_missing.genotypes Adds expected genotype truth set for X-chr functional scenario (no recombination + missingness).
tests/functional_tests/test_sex/ped_file-with_recom.txt Adds pedigree inputs (with recombination) including sex coding.
tests/functional_tests/test_sex/ped_file-with_recom_missing.txt Adds pedigree inputs (with recombination + missingness) including sex coding.
tests/functional_tests/test_sex/ped_file-no_recom.txt Adds pedigree inputs (no recombination) including sex coding.
tests/functional_tests/test_sex/ped_file-no_recom_missing.txt Adds pedigree inputs (no recombination + missingness) including sex coding.
tests/functional_tests/test_sex/geno_file-with_recom.txt Adds genotype inputs (with recombination) for X-chr functional test.
tests/functional_tests/test_sex/geno_file-with_recom_missing.txt Adds genotype inputs (with recombination + missingness) for X-chr functional test.
tests/functional_tests/test_sex/geno_file-no_recom.txt Adds genotype inputs (no recombination) for X-chr functional test.
tests/functional_tests/test_sex/geno_file-no_recom_missing.txt Adds genotype inputs (no recombination + missingness) for X-chr functional test.
tests/functional_tests/run_func_test.py Adds parametrized X-chr functional test invoking AlphaImpute2 with -x_chr.
tests/accuracy_tests/sim_for_alphapeel_accu_test/X_chr_ped_file.txt Adds simulated pedigree file for X-chr accuracy benchmarking.
tests/accuracy_tests/run_accu_test.py Extends accuracy benchmarking to support X-chr mode and new metrics output format.
src/alphaimpute2/Imputation/PhasingObjects.py Adds X-chr male consensus logic and plumbs sex/isXChr into consensus computation.
src/alphaimpute2/Imputation/ParticlePhasing.py Updates reference library and phasing to handle X-chr males (single haplotype, special consensus and haplotype assignment).
src/alphaimpute2/Imputation/ParticleImputation.py Passes x_chr into reference-library creation during imputation.
src/alphaimpute2/Imputation/ImputationIndividual.py Adds sex and isXChr to jit views; adds X-chr male penetrance/genotype-prob handling.
src/alphaimpute2/Imputation/Imputation.py Adds X-chr-specific genotype/phase fill/align helpers.
src/alphaimpute2/Imputation/Heuristic_Peeling.py Adds X-chr segregation/anterior/posterior handling and specialized smoothing/transmission logic.
src/alphaimpute2/alphaimpute2.py Adds -x_chr and -geno_prob CLI flags, improves version handling, and updates output writing.
pyproject.toml Updates author email and adjusts supported Python range metadata.
example/simple_example/simple_pedigree.txt Adds simple example pedigree input file to repo.
example/simple_example/simple_genotype.txt Adds simple example genotype input file to repo.
example/run_examples.sh Makes example runner more robust/documented and adds shebang.
example/data/X_chr/true_simple_segregation.txt Adds expected segregation truth for X-chr example.
example/data/X_chr/true_simple_haplotype.txt Adds expected haplotype truth for X-chr example.
example/data/X_chr/true_simple_genotype.txt Adds expected genotype truth for X-chr example.
example/data/X_chr/simple_X_chr.haplotypes Adds example X-chr phased haplotypes file.
example/data/X_chr/simple_X_chr.genotypes Adds example X-chr genotypes file.
example/data/X_chr/simple_pedigree.txt Adds example X-chr pedigree file including sex column.
example/data/X_chr/simple_genotype.txt Adds example X-chr genotype file including missingness patterns.
example/check_accuracy.r Improves formatting/output of R accuracy script and adds shebang.
docs/source/usage.rst Adds new detailed usage/file-format documentation including X-chr encoding guidance.
docs/source/introduction.rst Adds new introduction page for Sphinx docs.
docs/source/index.rst Refactors docs index into toctree-driven structure.
docs/source/get_started.rst Adds getting started/install/build/run guidance.
docs/source/conf.py Adds global Sphinx substitution for `
docs/source/changelog.rst Adds initial changelog scaffold.
docs/source/algorithm.rst Adds algorithm overview documentation (peeling terms, segregation, etc.).
conftest.py Reworks how accuracy metrics are summarized at end of pytest run.
.github/workflows/tests.yml Expands CI matrix to run across multiple Python versions.
.github/ISSUE_TEMPLATE/task-issue-template.md Adds task issue template.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread conftest.py
Comment thread conftest.py
Comment thread conftest.py
Comment thread conftest.py
Comment on lines +103 to +107
for row in file_data:
value = row["value"]
bar_length = int(
max_length * (value - min_value) / (max_value - min_value)
)
Comment on lines +258 to +266
def run_command(cmd):
exit_code = os.system(cmd)
if exit_code == 11:
import glob

outputs = glob.glob(os.path.join(output_path, "test.*"))
if outputs:
return # output was written, crash was in cleanup only
assert exit_code == 0, f"AlphaImpute2 failed with exit code {exit_code}"
Comment thread src/alphaimpute2/Imputation/ImputationIndividual.py
Comment on lines +386 to +389
if g == 2:
raise ValueError(
f"Unexpected genotype value for male in sex chr: {g}"
)
Comment on lines +729 to +732
if g == 2:
raise ValueError(
f"Unexpected genotype value for male in sex chr: {g}"
)
Comment on lines +525 to +537
if isXChr:
if child.sex == 0:
if geno == 2:
print(
f"Warning: No possible genotype 2 in male at position {i} of {child.idn} "
)
if geno == 1 and hap1 == 0:
print(
f"Warning: No possible genotype 1 and maternal haplotype is 0 in male at position {i} of {child.idn} "
)
return logGenotypeSegregationTensor_XYChrom[seg0, seg1, hap0, hap1, geno]
else:
return logGenotypeSegregationTensor_XXChrom[seg0, seg1, hap0, hap1, geno]
Comment thread src/alphaimpute2/alphaimpute2.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add x_chr functionality

4 participants