《统计学习方法》笔记——基于 Python 算法实现
This repository is an educational companion to classic statistical-learning methods. It explains the mathematics, implements the main ideas by hand in Python, and uses scikit-learn only for comparison cells that were part of the original notebooks.
The 2026 revival keeps that original purpose intact: the notebooks are learning material first, and a modern runtime is a way to make that material usable again—not a reason to replace the implementations with library calls.
The project was created on 28 December 2017 and its original development series ended on 9 January 2018. The original Chinese notes, examples, plots, and chapter order are preserved while the maintenance work adds reproducible setup, validation, and small compatibility fixes in separate pull requests.
| Chapter | Topic | Notebook |
|---|---|---|
| 1 | 最小二乘法 / Least squares | least_sqaure_method.ipynb |
| 2 | 感知机 / Perceptron | Iris_perceptron.ipynb |
| 3 | k 近邻法 / k-nearest neighbors | KNN.ipynb |
| 4 | 朴素贝叶斯 / Naive Bayes | GaussianNB.ipynb |
| 5 | 决策树 / Decision tree | DT.ipynb |
| 6 | 逻辑斯谛回归 / Logistic regression | LR.ipynb |
| 7 | 支持向量机 / SVM | support-vector-machine.ipynb |
| 8 | AdaBoost | Adaboost.ipynb |
| 9 | EM 算法 / Expectation-Maximization | em.ipynb |
The supported development range is Python 3.10–3.13. uv creates the environment from the committed lock file:
uv sync --all-groups
uv run python -m pytest
uv run python scripts/validate_notebooks.py
uv run python scripts/execute_notebooks.pyThe last command executes all nine notebooks that are currently compatible with modern dependencies. The validator independently checks every committed notebook for notebook-format and stored-error problems. Open the notebooks in JupyterLab with:
uv run jupyter labThe decision-tree notebook's final visualization renders when the system
Graphviz dot executable is available. Without it, the notebook completes
and reports the optional rendering dependency while retaining the DOT source.
The Python graphviz package is included in the environment; install the
executable with your platform's package manager when you want the image.
Branch: master after the 2026 revival promotion
Current stage: maintenance stack promoted; v2026.1.0 release tag pending
Original upstream baseline: last commit 9 January 2018; no published release
The following snapshot was collected on 12 August 2026 in a clean CPython
3.13.13 virtual environment using the latest versions resolved for this audit.
The committed uv.lock is the reproducibility source; this table records what
was actually exercised during the baseline audit.
| Component | Audit version |
|---|---|
| Python | 3.13.13 |
| NumPy | 2.5.2 |
| pandas | 3.0.5 |
| matplotlib | 3.11.1 |
| scikit-learn | 1.9.0 |
| Notebook | Baseline result | Finding / follow-up |
|---|---|---|
| Least squares | Pass | Smoke-tested; preserve the curve-fitting explanation and outputs. PR 2. |
| Perceptron | Pass | Hand-written SGD module, current sklearn comparison, and classifier tests added in PR 4. |
| k-nearest neighbors | Pass | Hand-written distance/voting module, deterministic split, and correctness tests added in PR 5. |
| Naive Bayes | Pass | Hand-written Gaussian module, 2-D sklearn single-sample comparison, and tests added in PR 6. |
| Decision tree | Pass | Hand-written ID3 module/tests added; Graphviz rendering is optional when dot is unavailable. PR 7. |
| Logistic regression | Pass | Hand-written stochastic-gradient implementation extracted and tested; NumPy-safe sigmoid and deterministic split added in PR 8. |
| SVM | Pass | model_selection.train_test_split migration complete; hand-written SMO module and tests added in PR 3. |
| AdaBoost | Pass | Hand-written threshold weak learners now handle perfect, constant, and coarse-step features; reproducible 100-run experiment added in PR 9. |
| EM | Pass | Generator-based Bernoulli-mixture implementation now captures fit data instead of reading global state; correctness tests added in PR 10. |
The baseline notebook metadata still identifies Python 3.6.1. That metadata is historical, not a supported runtime declaration. No notebook implementation behavior is changed in PR 1.
GitHub Actions validates all nine primary notebooks as nbformat 4 documents and
executes all nine current smoke notebooks on Python 3.10, 3.11, 3.12, and
3.13. The matrix does not hide notebook failures behind
allow_errors; compatibility notes remain in the table above.
- PR 1 — Foundation: MIT license, ignore rules, checkpoint cleanup, locked dependencies, notebook validation/smoke CI, README, and contribution guidance.
- PR 2 — Least squares: modern execution boundaries and lightweight numerical correctness tests.
- PR 3 — SVM: migrate deprecated imports, preserve the hand-written SMO explanation, and add kernel/classification checks.
- PR 4 — Perceptron: current sklearn comparison API and hand-written classifier tests.
- PR 5 — k-nearest neighbors: input validation, deterministic examples, and distance/prediction tests.
- PR 6 — Naive Bayes: modern single-sample comparison shape and tests for Gaussian probability/classification behavior.
- PR 7 — Decision tree: preserve the entropy/information-gain implementation, make the visualization path portable, and test tree predictions.
- PR 8 — Logistic regression: preserve gradient descent while making scalar/array behavior explicit and testing convergence on the teaching data.
- PR 9 — AdaBoost: retain the threshold weak learners, address runtime and edge cases, and test weight updates/classification.
- PR 10 — EM: preserve the generator-based teaching flow and add parameter/convergence tests.
- PR 11 — Release readiness: CHANGELOG.md, the 2026 revival release plan, final compatibility matrix, and issue/PR triage record.
At the initial audit time GitHub Issues were disabled and the repository reported zero pull requests, so there was no existing issue or PR backlog to merge into this plan. The scoped Draft PR chain above is now the public record of this revival work.
See CONTRIBUTING.md for the development workflow, notebook preservation rules, validation commands, and pull-request boundaries.
This project is released under the MIT License, copyright
wzyonggege.