Skip to content

Kaggle install cell: drop the unused pip3-autoremove install - #419

Draft
danielhanchen wants to merge 1 commit into
mainfrom
perf/kaggle-drop-autoremove
Draft

danielhanchen wants to merge 1 commit into
mainfrom
perf/kaggle-drop-autoremove

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

What changes

Every Kaggle notebook built from the shared install template (79 in nb/Kaggle-*.ipynb) starts with !pip install pip3-autoremove. No notebook, template or generator code ever runs pip-autoremove. The only other mentions of the name are two install-cell detectors in update_all_notebooks.py, and those are unchanged. This drops the line from the template.

Only update_all_notebooks.py was edited. The 79 Kaggle notebooks and their python_scripts/ mirrors were regenerated, and the commit includes only the files this change touches. Regenerating from this branch reproduces the committed files exactly, apart from drift that main shows too.

Results

These are real Kaggle kernel runs of the generated Kaggle-Qwen3_(14B) install cell.

  • The removed line took 4.5, 5.4, 4.1 and 3.7 s across four runs on main (median 4.3 s).
  • The cell runs one fewer pip transaction (5 to 4) and installs one fewer distribution (17 to 16), the same in every run.
  • The final pip list --format=freeze is identical to main except for pip3-autoremove==2.1.0, which is no longer installed. That held in 3 of 3 runs. Its dependencies (pip, packaging) were already present.
  • Whole-cell wall time was 72.3 / 72.9 / 98.7 s on main and 45.8 / 72.4 / 79.1 s here. The difference is within run-to-run noise, so the per-line number above is the measurement to use.

Tests

  • No generated notebook installs pip3-autoremove. This fails on main (79 notebooks) and passes here.
  • The repo test suite gives 12 failed / 15878 passed / 582 skipped on both main and this branch, with the same failing test IDs. All 12 are in test_notebook_api_drift.py and need unsloth importable.

Not covered

  • The Kaggle runs used CPU kernels because the GPU quota was used up for the week. The line being removed does the same work on either image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant