Refactor/refactor AlphaPeel by XingerTang · Pull Request #292 · AlphaGenes/AlphaPeel

XingerTang · 2026-07-07T17:57:24Z

Related Issue

Closes #78
Closes #172
Closes #178
Closes #258
Working...

What changed

Call modules of functional tests directly instead of building, installing, and running with an OS command
Check directly against the error message and warning message in the functional tests
Improve the readability of the functional test file
Use ValueError instead of printing error messages
Add instructions for functional tests memory profiling and coverage report generation
Simplify the accuracy test so that it only runs minimal
Provide separate full performance benchmarking in utility files.
Provide visualization of the full benchmarking report.

Working...

Bigger picture of the full plan

Modification on tests:
- For functional tests:
  - Call modules directly instead of build and install and run with OS command
  - It would enable:
    - memory profiling
    - function-wise runtime profiling
    - test coverage report
    - more informative debugging information when test failed
    - catch errors and warnings easier than the other way
- Keep the accuracy tests as the current build and install approach to test by platform and distribution
- Update accuracy test table as AlphaImpute2
Refactor the reassignment of argument options in AlphaPeel to remove warnings raised because of the unsynced arguments retrieval.
Remove unused/commented code
Replace single-letter variable names with more informative names
Add comments and docstrings when applicable
Memory optimisation
- Background: For a 2MB genotype input, the PeelingInfo would take 174MB
- By reviewing the current most memory-demanding data, several improvements can make:
  - PointSeg is used a temporary holder of segregation probabilities of one single individual, no necessity to allocate it an nInd x 4 x nLoci array.
  - Similar case but a little bit more complicated situation may apply to family terms:
    - posteriorSire_new
    - posteriorDam_new
    - posteriorSire_minusFam
    - posteriorDam_minusFam
  - There are several variables of int64 type, int64 is too large than what we need
Move the functions that are jit-compilable to jit
Explore multi-processing solution on HPC

Why this change

Refactoring the code

Working...

Submodule changes

Submodule repository: AlphaGenes/tinyhouse
PR: Refactor/refactor alphapeel tinyhouse#40
Why needed: argparsing, file I/O, ...

Notes / Risks

Working...

XingerTang · 2026-07-07T18:02:25Z

Coverage report of the functional test:

coverage report -m
Name                                            Stmts   Miss  Cover   Missing
-----------------------------------------------------------------------------
conftest.py                                       104     33    68%   10-18, 23-33, 58-71, 150-151, 198
src/tinypeel/Peeling/Peeling.py                   268    240    10%   31-243, 268-274, 292-303, 324-332, 352-360, 370-382, 412-427, 446-452, 471-477, 490-512, 525-542, 565-641
src/tinypeel/Peeling/PeelingIO.py                 173     68    61%   24-84, 151-152, 183, 191, 214, 267-285, 430-433, 447-450, 463-487
src/tinypeel/Peeling/PeelingInfo.py               211     74    65%   54, 153, 210-213, 232-235, 269, 289-300, 366-383, 391-434, 446-458, 470-482
src/tinypeel/Peeling/PeelingUpdates.py            191     74    61%   35, 54, 75, 77, 88, 90, 95, 113-135, 153-164, 261, 279, 336-344, 397-400, 413-437, 462-466, 503-509, 524-547, 560-564, 576
src/tinypeel/Peeling/version.py                     2      0   100%
src/tinypeel/tinyhouse/HaplotypeOperations.py      59     42    29%   6-8, 16-22, 31-39, 44-46, 51-54, 59-63, 68-73, 78-85
src/tinypeel/tinyhouse/InputOutput.py             343    156    55%   15, 42-62, 70-71, 76-77, 87-88, 98-106, 114-122, 243-258, 262-284, 308-309, 327-328, 331-340, 343, 357-358, 368-370, 415-416, 438-446, 453-454, 458-459, 481-482, 485-486, 510-567, 576-577, 590-604, 610-633, 639-642, 646-655, 659-666, 698, 700
src/tinypeel/tinyhouse/MultiThreadIO.py            78     44    44%   9-10, 15-16, 21-47, 52, 56-63, 97-105, 122-138, 151
src/tinypeel/tinyhouse/Pedigree.py                778    350    55%   45-48, 107, 110, 120-137, 143, 199, 212-214, 292, 300-310, 313-337, 342-363, 366, 402-408, 411-418, 422-427, 434, 437, 447-451, 483, 501-512, 530-557, 560-567, 665, 675, 677, 680-708, 722, 728, 732, 740, 767-789, 795-798, 805-824, 829-849, 855-872, 883-938, 981, 986-987, 1009-1021, 1091, 1097, 1104, 1116-1138, 1146, 1154-1159, 1176-1177, 1181, 1185-1186, 1193-1199, 1212, 1215-1226, 1229-1237, 1240-1244, 1247-1252, 1255-1264, 1270-1279, 1283-1294, 1298-1307, 1310, 1315-1317, 1322-1325, 1330-1332, 1337-1340, 1345-1348
src/tinypeel/tinyhouse/ProbMath.py                251    115    54%   35-55, 65, 67, 95-96, 106-140, 145-147, 152-164, 176-211, 215-254, 260-262, 267-270, 275-278, 283-289
src/tinypeel/tinypeel.py                          286     50    83%   97, 113-119, 146-149, 165-168, 300-327, 350-362, 812-813, 845, 849-855, 886-891, 899
tests/functional_tests/run_func_test.py           527      2    99%   79, 177
-----------------------------------------------------------------------------
TOTAL                                            3271   1248    62%

gregorgorjanc

The changes look good/ok

XingerTang · 2026-07-08T11:34:51Z

Functional tests memory profiling - this is not very informative, as the Numba function can be called at any part of the run:

======================================================================================= MEMRAY REPORT =======================================================================================
Allocation results for tests/functional_tests/run_func_test.py::TestClass::test_out_id_only at the high watermark

         📦 Total memory allocated: 90.3MiB
         📏 Total allocations: 899
         📊 Histogram of allocation sizes: | ▂█▂▁|
         🥇 Biggest allocating functions:
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 19.6MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 4.6MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 4.3MiB
                - _to_string:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/ir/values.py:549 -> 2.5MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 2.1MiB


Allocation results for tests/functional_tests/run_func_test.py::TestClass::test_files at the high watermark

         📦 Total memory allocated: 52.2MiB
         📏 Total allocations: 3811
         📊 Histogram of allocation sizes: |  █  |
         🥇 Biggest allocating functions:
                - _compile_bytecode:<frozen importlib._bootstrap_external>:729 -> 2.9MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 2.2MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 2.1MiB
                - _compile_bytecode:<frozen importlib._bootstrap_external>:729 -> 1.1MiB
                - updatecache:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/linecache.py:137 -> 1.1MiB


Allocation results for tests/functional_tests/run_func_test.py::TestClass::test_est at the high watermark

         📦 Total memory allocated: 9.4MiB
         📏 Total allocations: 482
         📊 Histogram of allocation sizes: | ▁█▄ |
         🥇 Biggest allocating functions:
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 1.2MiB
                - _icmp:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/ir/builder.py:575 -> 1.0MiB
                - init_first_block:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/numba/core/interpreter.py:1519 -> 1.0MiB
                - __init__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/ir/instructions.py:464 -> 1.0MiB
                - generic:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/numba/core/typing/npydecl.py:97 -> 1.0MiB


Allocation results for tests/functional_tests/run_func_test.py::TestClass::test_pheno at the high watermark

         📦 Total memory allocated: 9.0MiB
         📏 Total allocations: 397
         📊 Histogram of allocation sizes: |  █▁ |
         🥇 Biggest allocating functions:
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 3.4MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 2.0MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 604.1KiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 415.7KiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 170.1KiB


Allocation results for tests/functional_tests/run_func_test.py::TestClass::test_sex at the high watermark

         📦 Total memory allocated: 4.2MiB
         📏 Total allocations: 316
         📊 Histogram of allocation sizes: | ▁█▂ |
         🥇 Biggest allocating functions:
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 2.6MiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 185.3KiB
                - register:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/ir/_utils.py:21 -> 128.0KiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 77.2KiB
                - __call__:/Users/xtang3/anaconda3/envs/phase/lib/python3.11/site-packages/llvmlite/binding/ffi.py:197 -> 77.2KiB

XingerTang · 2026-07-08T11:42:41Z

Coverage report of the functional test:

coverage report -m
Name                                            Stmts   Miss  Cover   Missing
-----------------------------------------------------------------------------
conftest.py                                       104     33    68%   10-18, 23-33, 58-71, 150-151, 198
src/tinypeel/Peeling/Peeling.py                   268    240    10%   31-243, 268-274, 292-303, 324-332, 352-360, 370-382, 412-427, 446-452, 471-477, 490-512, 525-542, 565-641
src/tinypeel/Peeling/PeelingIO.py                 173     68    61%   24-84, 151-152, 183, 191, 214, 267-285, 430-433, 447-450, 463-487
src/tinypeel/Peeling/PeelingInfo.py               211     74    65%   54, 153, 210-213, 232-235, 269, 289-300, 366-383, 391-434, 446-458, 470-482
src/tinypeel/Peeling/PeelingUpdates.py            191     74    61%   35, 54, 75, 77, 88, 90, 95, 113-135, 153-164, 261, 279, 336-344, 397-400, 413-437, 462-466, 503-509, 524-547, 560-564, 576
src/tinypeel/Peeling/version.py                     2      0   100%
src/tinypeel/tinyhouse/HaplotypeOperations.py      59     42    29%   6-8, 16-22, 31-39, 44-46, 51-54, 59-63, 68-73, 78-85
src/tinypeel/tinyhouse/InputOutput.py             343    156    55%   15, 42-62, 70-71, 76-77, 87-88, 98-106, 114-122, 243-258, 262-284, 308-309, 327-328, 331-340, 343, 357-358, 368-370, 415-416, 438-446, 453-454, 458-459, 481-482, 485-486, 510-567, 576-577, 590-604, 610-633, 639-642, 646-655, 659-666, 698, 700
src/tinypeel/tinyhouse/MultiThreadIO.py            78     44    44%   9-10, 15-16, 21-47, 52, 56-63, 97-105, 122-138, 151
src/tinypeel/tinyhouse/Pedigree.py                778    350    55%   45-48, 107, 110, 120-137, 143, 199, 212-214, 292, 300-310, 313-337, 342-363, 366, 402-408, 411-418, 422-427, 434, 437, 447-451, 483, 501-512, 530-557, 560-567, 665, 675, 677, 680-708, 722, 728, 732, 740, 767-789, 795-798, 805-824, 829-849, 855-872, 883-938, 981, 986-987, 1009-1021, 1091, 1097, 1104, 1116-1138, 1146, 1154-1159, 1176-1177, 1181, 1185-1186, 1193-1199, 1212, 1215-1226, 1229-1237, 1240-1244, 1247-1252, 1255-1264, 1270-1279, 1283-1294, 1298-1307, 1310, 1315-1317, 1322-1325, 1330-1332, 1337-1340, 1345-1348
src/tinypeel/tinyhouse/ProbMath.py                251    115    54%   35-55, 65, 67, 95-96, 106-140, 145-147, 152-164, 176-211, 215-254, 260-262, 267-270, 275-278, 283-289
src/tinypeel/tinypeel.py                          286     50    83%   97, 113-119, 146-149, 165-168, 300-327, 350-362, 812-813, 845, 849-855, 886-891, 899
tests/functional_tests/run_func_test.py           527      2    99%   79, 177
-----------------------------------------------------------------------------
TOTAL                                            3271   1248    62%

Coverage report cannot identify the calls to jit-compiled functions.

XingerTang · 2026-07-08T20:18:07Z

The original accuracy tests have been split into two parts:

Minimalist accuracy test for CI/CD quality check
- Simplified report so we can focus on the most important values during production
- Much faster and lighter, so shorter delay for accuracy checks
- Still use the built distribution version for the OS-wise check
Accuracy benchmarking for monitoring the performance of combinations of different options
- Maximised report to cover as many details as possible
- Call the modules directly so that it runs faster and captures the debugging info/profiling info better
  - Run by from src.utils import run_full_accuracy_suite as run_full; run_full()

Still under development...

XingerTang · 2026-07-09T12:52:35Z

GitHub runner does not support Python 3.7 for ubuntu:

Error: The version '3.7' with architecture 'x64' was not found for Ubuntu 24.04.

for simplicity, just test version 3.9 to 3.13.
(AlphaImpute2 cross version testing is wrong, and requires a fix.)

…hr output

XingerTang · 2026-07-10T16:58:23Z

The full report has 439 lines, and it is impossible to plot all the details; the plots generated by the new code include only marker_corr, abs_diff, correct_rate and elapsed_seconds(runtime). But users can change the code to plot the other metrics.

generation_profile_abs_diff_phased_geno_prob

generation_profile_correct_rate_seg_prob

generation_profile_marker_corr_geno_0_333

generation_profile_marker_corr_geno_prob

generation_profile_marker_corr_phased_geno_prob

XingerTang · 2026-07-10T17:05:00Z

The full report:
accu_report.txt

gregorgorjanc · 2026-07-10T17:56:53Z

This looks very comprehensive @XingerTang! Once question though - say I make changes and then open PR - and will get these plots generated, which is great, but really need to seek for differences from the previous implementation - how can we address that aspect!?

XingerTang · 2026-07-10T18:16:24Z

This looks very comprehensive @XingerTang! Once question though - say I make changes and then open PR - and will get these plots generated, which is great, but really need to seek for differences from the previous implementation - how can we address that aspect!?

There are no functions for comparison across different implementations yet. But a simple way to do that is to copy a accu_report.txt before the change, then later generate a new one and make a diff against the two if we don't expect too many changes.

Visualisations for comparisons between different accu_report.txt are possible and probably quite useful but are not my current focus.

XingerTang · 2026-07-10T18:19:46Z

The commit message for 897287f is wrong, but I don't want to do force push to correct it, just make a note here, the commit splits the utils.py for code readability, and update documentation.

gregorgorjanc · 2026-07-11T21:13:35Z

There are no functions for comparison across different implementations yet. But a simple way to do that is to copy a accu_report.txt before the change, then later generate a new one and make a diff against the two if we don't expect too many changes.

Please make a note of this in the docs for developers.

XingerTang · 2026-07-13T10:58:49Z

AlphaPeel has an unignorable amount of uncalled haplotypes by default, as shown in the accuracy test report:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ uncalled_rate ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The number of loci with haplotypes uncalled divided by the total number of loci.
----------------------------------- hap_0.5 ------------------------------------
Method               Value     
--------------------------------------------------------------------------------
single               0.225 | ###########################################............. |
multi                0.104 | ........................................................ |
hybrid               0.261 | ######################################################## |

The uncalled haplotypes distort the values of other metrics in the report. The recent commit ignores all the uncalled haplotypes in the calculation. Now the heatmaps are:

gregorgorjanc · 2026-07-13T11:54:26Z

@XingerTang interesting - why do we see/get this? Isn't the 0.5 threshold forcing the call even if we are quite uncertain or is it that the probs are exactly 0.5 for these loci?

XingerTang · 2026-07-13T11:59:06Z

@XingerTang interesting - why do we see/get this? Isn't the 0.5 threshold forcing the call even if we are quite uncertain or is it that the probs are exactly 0.5 for these loci?

The phased genotypes can be [0.4999, 0.4999, 0.0001, 0.0001] and this won't be called with hap_threshold 0.5.

XingerTang · 2026-07-13T12:59:29Z

In the recent commits, the argument options are regrouped according to the usage section of the documentation.

The original help message is:

(phase) xtang3@S37-50MAQ05D AlphaPeel % AlphaPeel
------------------------------------------
                AlphaPeel                 
------------------------------------------
Version: 1.3.0
Docs:    https://alphapeel.readthedocs.io/en/v1.3.0/usage.html
Email:   alphagenes.dev@gmail.com
Website: https://github.com/AlphaGenes
------------------------------------------
usage: AlphaPeel [-h] -out_file OUT_FILE [-method METHOD] [-geno_file [GENO_FILE ...]] [-pheno_file [PHENO_FILE ...]] [-hap_file [HAP_FILE ...]] [-seq_file [SEQ_FILE ...]]
                 [-ped_file [PED_FILE ...]] [-alt_allele_prob_file [ALT_ALLELE_PROB_FILE ...]] [-pheno_penetrance_prob_file [PHENO_PENETRANCE_PROB_FILE ...]] [-start_snp START_SNP]
                 [-stop_snp STOP_SNP] [-main_metafounder MAIN_METAFOUNDER] [-map_file MAP_FILE] [-phased_geno_prob_file [PHASED_GENO_PROB_FILE ...]] [-no_dosage] [-geno]
                 [-geno_threshold [GENO_THRESHOLD ...]] [-geno_prob] [-phased_geno_prob] [-hap] [-hap_threshold [HAP_THRESHOLD ...]] [-seg_prob] [-pheno_prob] [-rec_prob] [-alt_allele_prob]
                 [-pheno_penetrance_prob] [-out_id_order OUT_ID_ORDER] [-out_id_only] [-out_digits OUT_DIGITS] [-n_thread N_THREAD] [-rec_length REC_LENGTH] [-n_cycle N_CYCLE]
                 [-geno_error_prob GENO_ERROR_PROB] [-seq_error_prob SEQ_ERROR_PROB] [-mut_prob MUT_PROB] [-x_chr] [-est_start_alt_allele_prob] [-est_alt_allele_prob] [-est_geno_error_prob]
                 [-est_seq_error_prob] [-est_pheno_penetrance_prob] [-no_phase_founder] [-seg_map_file SEG_MAP_FILE] [-seg_file SEG_FILE] [-version]

options:
  -h, --help            show this help message and exit
  -version              Show program's version number and exit.

Core arguments:
  -out_file OUT_FILE    The output file prefix. All file outputs will be named as "PREFIX.OUTPUT.txt", where "OUTPUT" is the type of output (for example, "dosage" and "geno_prob").

Mandatory peeling arguments:
  -method METHOD        Peeling method: single or multi. Default: multi.

Input Options:
  -geno_file [GENO_FILE ...]
                        Genotype file(s) (see format details in the docs).
  -pheno_file [PHENO_FILE ...]
                        Phenotype file(s) (see format details in the docs).
  -hap_file [HAP_FILE ...]
                        Haplotype file (see format details in the docs).
  -seq_file [SEQ_FILE ...]
                        Sequence allele read count file(s) (see format details in the docs).
  -ped_file [PED_FILE ...]
                        Pedigree file(s) in (see format details in the docs).
  -alt_allele_prob_file [ALT_ALLELE_PROB_FILE ...]
                        Alternative allele probability file (see format details in the docs). Default: 0.5 for each locus.
  -pheno_penetrance_prob_file [PHENO_PENETRANCE_PROB_FILE ...]
                        Phenotype penetrance probability file (see format details in the docs).
  -start_snp START_SNP  The first locus to consider. Counting starts at 1. Default: 1.
  -stop_snp STOP_SNP    The last locus to consider. Default: all loci in input files
  -main_metafounder MAIN_METAFOUNDER
                        ID used to represent the base population of the pedigree (metafounder / unknown parent group) (see format details in the docs). Default: MF_1.
  -map_file MAP_FILE    Map file for loci in genomic data files (see format details in the docs).
  -phased_geno_prob_file [PHASED_GENO_PROB_FILE ...]
                        Optional external phased genotype probability file(s) (see format details in the docs). This will provide the starting internal genotype probability state.

Output Options:
  -no_dosage            Suppress default output of allele dosages (see format details in the docs).
  -geno                 Call and output genotypes (see format details in the docs). The default genotype calling threshold is set to 1/3.
  -geno_threshold [GENO_THRESHOLD ...]
                        Custom genotype calling threshold(s) from the genotype probabilities. Multiple space separated values allowed. Value(s) less than 1/3 are replaced by 1/3.
  -geno_prob            Output genotype probabilities (see format details in the docs).
  -phased_geno_prob     Output phased genotype probabilities (see format details in the docs).
  -hap                  Call and output haplotypes (see format details in the docs). The default haplotype calling threshold is set to 1/2.
  -hap_threshold [HAP_THRESHOLD ...]
                        Custom haplotype calling threshold(s) from the phased genotype probabilities. Multiple space separated values allowed. Value(s) less than 1/2 are replaced by 1/2.
  -seg_prob             Output segregation probabilities (see format details in the docs).
  -pheno_prob           Output phenotype probabilities (see format details in the docs).
  -rec_prob             Output recombination probabilities (NOT IMPLEMENTED YET).
  -alt_allele_prob      Output alternative allele frequencies (see format details in the docs). Output 0.5 if none of ``est_start_alt_allele_prob``, ``est_alt_allele_prob``, or
                        ``alt_allele_prob_file`` is used.
  -pheno_penetrance_prob
                        Output phenotype penetrance probabilities (see format details in the docs).
  -out_id_order OUT_ID_ORDER
                        Determines the order in which individuals are ordered in the output file based on their order in the corresponding input file. Individuals not in the input file are
                        placed at the end of the file and sorted in alphanumeric order. These inividuals can be surpressed with the "-out_id_only" option. Options: id, pedigree, genotypes,
                        sequence, segregation. Defualt: id.
  -out_id_only          Suppress output for individuals not present in the file specified with -out_id_order. It also suppresses "dummy" individuals.
  -out_digits OUT_DIGITS
                        Specify the number of digits to round the outputs. Does not apply to outputs from ``alt_allele_prob``, ``geno_error_prob``, ``seq_error_prob``, and ``pheno_penetrance``.
                        Default: 4.

Multithreading Options:
  -n_thread N_THREAD    Maximum number of threads to use. Default: 1.

Optional peeling arguments:
  -rec_length REC_LENGTH
                        Recombination length of the chromosome in Morgans. Default: 1.00
  -n_cycle N_CYCLE      Number of peeling cycles. Default: 5.
  -geno_error_prob GENO_ERROR_PROB
                        Genotype error probability. Default: 0.0001.
  -seq_error_prob SEQ_ERROR_PROB
                        Sequencing error probability. Must not be 0. Default: 0.001.
  -mut_prob MUT_PROB    Mutation probability. Default: 1e-8.

Peeling control arguments:
  -x_chr                Indicate that input data is for the X chromosome (see details in the docs).
  -est_start_alt_allele_prob
                        Estimate from all inputted genomic data prior to peeling and output alternative allele probabilities (see format details in the docs).
  -est_alt_allele_prob  Estimate after each peeling cycle and output alternative allele probabilities (see format details in the docs).
  -est_geno_error_prob  Estimate after each peeling cycle and output genotype error probabilities (see format details in the docs).
  -est_seq_error_prob   Estimate after each peeling cycle and output sequence error probabilities (see format details in the docs).
  -est_pheno_penetrance_prob
                        Estimate after each peeling cycle and output phenotype penetrance probabilities (see format details in the docs).
  -no_phase_founder     Suppress phasing a heterozygous allele (if such an allele can be found) in genotyped individuals without genotyped parents.

Hybrid peeling arguments:
  -seg_map_file SEG_MAP_FILE
                        Map file for loci in the segregation probabilities file.
  -seg_file SEG_FILE    Segregation probabilities file (see format details in the docs).

The help message after change:

(phase) xtang3@S37-50MAQ05D AlphaPeel % AlphaPeel -h
------------------------------------------
              AlphaPeel                 
------------------------------------------
Version: 1.3.0
Docs:    https://alphapeel.readthedocs.io/en/v1.3.0/usage.html
Email:   alphagenes.dev@gmail.com
Website: https://github.com/AlphaGenes
------------------------------------------
usage: AlphaPeel [-h] [-version] [-ped_file [PED_FILE ...]] [-geno_file [GENO_FILE ...]] [-seq_file [SEQ_FILE ...]]
               [-hap_file [HAP_FILE ...]] [-pheno_file [PHENO_FILE ...]] [-x_chr] [-phased_geno_prob_file [PHASED_GENO_PROB_FILE ...]]
               [-map_file MAP_FILE] [-start_snp START_SNP] [-stop_snp STOP_SNP] [-alt_allele_prob_file [ALT_ALLELE_PROB_FILE ...]]
               [-main_metafounder MAIN_METAFOUNDER] [-pheno_penetrance_prob_file [PHENO_PENETRANCE_PROB_FILE ...]]
               [-rec_length REC_LENGTH] [-mut_prob MUT_PROB] [-geno_error_prob GENO_ERROR_PROB] [-seq_error_prob SEQ_ERROR_PROB]
               [-no_dosage] [-geno] [-geno_threshold [GENO_THRESHOLD ...]] [-geno_prob] [-phased_geno_prob] [-hap]
               [-hap_threshold [HAP_THRESHOLD ...]] [-seg_prob] [-pheno_prob] [-rec_prob] [-alt_allele_prob] [-pheno_penetrance_prob]
               -out_file OUT_FILE [-out_id_order OUT_ID_ORDER] [-out_id_only] [-out_digits OUT_DIGITS] [-method METHOD]
               [-seg_file SEG_FILE] [-seg_map_file SEG_MAP_FILE] [-n_cycle N_CYCLE] [-n_thread N_THREAD] [-est_start_alt_allele_prob]
               [-est_alt_allele_prob] [-est_geno_error_prob] [-est_seq_error_prob] [-est_pheno_penetrance_prob] [-no_phase_founder]

Program options:
-h, -help, --help     Show this help message and exit.
-version              Show program's version number and exit.

Input options: individuals:
-ped_file [PED_FILE ...]
                      Pedigree file(s) in (see format details in the docs).
-geno_file [GENO_FILE ...]
                      Genotype file(s) (see format details in the docs).
-seq_file [SEQ_FILE ...]
                      Sequence allele read count file(s) (see format details in the docs).
-hap_file [HAP_FILE ...]
                      Haplotype file (see format details in the docs).
-pheno_file [PHENO_FILE ...]
                      Phenotype file(s) (see format details in the docs).
-x_chr                Indicate that input data is for the X chromosome (see details in the docs).
-phased_geno_prob_file [PHASED_GENO_PROB_FILE ...]
                      Optional external phased genotype probability file(s) (see format details in the docs). This will provide the
                      starting internal genotype probability state.

Input options: markers:
-map_file MAP_FILE    Map file for loci in genomic data files (see format details in the docs).
-start_snp START_SNP  The first locus to consider. Counting starts at 1. Default: 1.
-stop_snp STOP_SNP    The last locus to consider. Default: all loci in input files

Input options: model parameters and other:
-alt_allele_prob_file [ALT_ALLELE_PROB_FILE ...]
                      Alternative allele probability file (see format details in the docs). Default: 0.5 for each locus.
-main_metafounder MAIN_METAFOUNDER
                      ID used to represent the base population of the pedigree (metafounder / unknown parent group) (see format
                      details in the docs). Default: MF_1.
-pheno_penetrance_prob_file [PHENO_PENETRANCE_PROB_FILE ...]
                      Phenotype penetrance probability file (see format details in the docs).
-rec_length REC_LENGTH
                      Recombination length of the chromosome in Morgans. Default: 1.00
-mut_prob MUT_PROB    Mutation probability. Default: 1e-8.
-geno_error_prob GENO_ERROR_PROB
                      Genotype error probability. Default: 0.0001.
-seq_error_prob SEQ_ERROR_PROB
                      Sequencing error probability. Must not be 0. Default: 0.001.

Output options: individuals:
-no_dosage            Suppress default output of allele dosages (see format details in the docs).
-geno                 Call and output genotypes (see format details in the docs). The default genotype calling threshold is set to
                      1/3.
-geno_threshold [GENO_THRESHOLD ...]
                      Custom genotype calling threshold(s) from the genotype probabilities. Multiple space separated values allowed.
                      Value(s) less than 1/3 are replaced by 1/3.
-geno_prob            Output genotype probabilities (see format details in the docs).
-phased_geno_prob     Output phased genotype probabilities (see format details in the docs).
-hap                  Call and output haplotypes (see format details in the docs). The default haplotype calling threshold is set to
                      1/2.
-hap_threshold [HAP_THRESHOLD ...]
                      Custom haplotype calling threshold(s) from the phased genotype probabilities. Multiple space separated values
                      allowed. Value(s) less than 1/2 are replaced by 1/2.
-seg_prob             Output segregation probabilities (see format details in the docs).
-pheno_prob           Output phenotype probabilities (see format details in the docs).
-rec_prob             Output recombination probabilities (NOT IMPLEMENTED YET).
-alt_allele_prob      Output alternative allele frequencies (see format details in the docs). Output 0.5 if none of
                      ``est_start_alt_allele_prob``, ``est_alt_allele_prob``, or ``alt_allele_prob_file`` is used.
-pheno_penetrance_prob
                      Output phenotype penetrance probabilities (see format details in the docs).

Output options: prefix, order, and IO:
-out_file OUT_FILE    The output file prefix. All file outputs will be named as "PREFIX.OUTPUT.txt", where "OUTPUT" is the type of
                      output (for example, "dosage" and "geno_prob").
-out_id_order OUT_ID_ORDER
                      Determines the order in which individuals are ordered in the output file based on their order in the
                      corresponding input file. Individuals not in the input file are placed at the end of the file and sorted in
                      alphanumeric order. These inividuals can be surpressed with the "-out_id_only" option. Options: id, pedigree,
                      genotypes, sequence, segregation. Defualt: id.
-out_id_only          Suppress output for individuals not present in the file specified with -out_id_order. It also suppresses "dummy"
                      individuals.
-out_digits OUT_DIGITS
                      Specify the number of digits to round the outputs. Does not apply to outputs from ``alt_allele_prob``,
                      ``geno_error_prob``, ``seq_error_prob``, and ``pheno_penetrance``. Default: 4.

Peeling methods: strategy:
-method METHOD        Peeling method: single or multi. Default: multi.

Peeling methods: hybrid second stage:
-seg_file SEG_FILE    Segregation probabilities file (see format details in the docs).
-seg_map_file SEG_MAP_FILE
                      Map file for loci in the segregation probabilities file.

Peeling parameters: computational:
-n_cycle N_CYCLE      Number of peeling cycles. Default: 5.
-n_thread N_THREAD    Maximum number of threads to use. Default: 1.

Peeling parameters: model estimation:
-est_start_alt_allele_prob
                      Estimate from all inputted genomic data prior to peeling and output alternative allele probabilities (see format
                      details in the docs).
-est_alt_allele_prob  Estimate after each peeling cycle and output alternative allele probabilities (see format details in the docs).
-est_geno_error_prob  Estimate after each peeling cycle and output genotype error probabilities (see format details in the docs).
-est_seq_error_prob   Estimate after each peeling cycle and output sequence error probabilities (see format details in the docs).
-est_pheno_penetrance_prob
                      Estimate after each peeling cycle and output phenotype penetrance probabilities (see format details in the
                      docs).
-no_phase_founder     Suppress phasing a heterozygous allele (if such an allele can be found) in genotyped individuals without
                      genotyped parents.

XingerTang · 2026-07-13T15:47:30Z

The unused functions are determined in combinations of:

call graph
coverage report
vulture package
code logic

XingerTang · 2026-07-13T15:55:03Z

All commented code is removed except the code with einsum notation, which is useful in the interpretation of the current code.

Refactoring functional tests

eb8cdff

XingerTang marked this pull request as draft July 7, 2026 17:57

gregorgorjanc approved these changes Jul 7, 2026

View reviewed changes

Fix pytest warnings

0ef79a9

Simplify accuracy test report

5bd3c0c

gregorgorjanc approved these changes Jul 8, 2026

View reviewed changes

XingerTang added 3 commits July 9, 2026 13:26

Proper hybrid test

c187b62

Update test workflow

ced4322

Shorten the list of Python versions

4712661

Update flake8 configuration

eb1a6f2

gregorgorjanc approved these changes Jul 9, 2026

View reviewed changes

XingerTang added 6 commits July 9, 2026 16:10

Test older version Python with older OS images

583252b

Python 3.6 is not well-supported

b64d72b

Update requirement

3ea4652

Limit support to python >= 3.8

6360ffb

Fix rounding for alt_allele_prob output

58ff958

Ignore first two rows of seg files & special case for hap file of x_c…

badef09

…hr output

gregorgorjanc approved these changes Jul 9, 2026

View reviewed changes

XingerTang added 2 commits July 10, 2026 12:34

Add gen-wise abs_diff and correct_rate output of seg_prob

1f327ac

Refactor utils.py

6ded945

gregorgorjanc approved these changes Jul 10, 2026

View reviewed changes

Add SER and relevant metrics for hap

781f5c0

gregorgorjanc approved these changes Jul 10, 2026

View reviewed changes

Add runtime and visualization

9ed5406

gregorgorjanc approved these changes Jul 10, 2026

View reviewed changes

Add SER and relevant metrics for hap

897287f

Ignore uncalled hap & update docs on accu report

8216678

XingerTang added 2 commits July 13, 2026 13:05

Refactor argument reassignment

28ef580

Regrouped the argument option parser

27ee3c5

gregorgorjanc approved these changes Jul 13, 2026

View reviewed changes

Remove unused functions

22c53a7

Remove commented code

782a144

gregorgorjanc reviewed Jul 13, 2026

View reviewed changes

Comment thread src/tinypeel/Peeling/PeelingUpdates.py

gregorgorjanc approved these changes Jul 13, 2026

View reviewed changes

Uh oh!

Conversation

XingerTang commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issue

What changed

Bigger picture of the full plan

Why this change

Submodule changes

Notes / Risks

Uh oh!

XingerTang commented Jul 7, 2026

Uh oh!

gregorgorjanc left a comment

Choose a reason for hiding this comment

Uh oh!

XingerTang commented Jul 8, 2026

Uh oh!

XingerTang commented Jul 8, 2026

Uh oh!

XingerTang commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

XingerTang commented Jul 9, 2026

Uh oh!

XingerTang commented Jul 10, 2026

Uh oh!

XingerTang commented Jul 10, 2026

Uh oh!

gregorgorjanc commented Jul 10, 2026

Uh oh!

XingerTang commented Jul 10, 2026

Uh oh!

XingerTang commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gregorgorjanc commented Jul 11, 2026

Uh oh!

XingerTang commented Jul 13, 2026

Uh oh!

gregorgorjanc commented Jul 13, 2026

Uh oh!

XingerTang commented Jul 13, 2026

Uh oh!

XingerTang commented Jul 13, 2026

Uh oh!

XingerTang commented Jul 13, 2026

Uh oh!

XingerTang commented Jul 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

XingerTang commented Jul 7, 2026 •

edited

Loading

XingerTang commented Jul 8, 2026 •

edited

Loading

XingerTang commented Jul 10, 2026 •

edited

Loading