Data onboarding: remove hardcoded paths, S3 dataset fetch, nuPlan docs - #525

Merged

eugenevinitsky merged 31 commits into

3.0from

ev/s3-data-fetch

Jul 24, 2026

Commits on Jul 10, 2026

Remove hardcoded /scratch nuPlan paths from shipped drive.ini

Eugene Vinitsky
and
claude
committed
6b3e25f
Document nuPlan download, conversion, and replay usage

Eugene Vinitsky
and
claude
committed
aa19a51
Remove AGENTIC_PR file

Eugene Vinitsky
and
claude
committed
c4a3aeb
Remove AGENTIC_PR file

Eugene Vinitsky
and
claude
committed
bf9f0ec
Add S3 dataset registry and fetch script

Eugene Vinitsky
and
claude
committed
1dabb9e
Fill in the nuPlan bin-creation pipeline: py123d + 123Drive

Eugene Vinitsky
and
claude
committed
71b9701
Point README Data section at the S3 fetch script

Eugene Vinitsky
and
claude
committed
029ab2c
Support credential-free fetch of public datasets

Eugene Vinitsky
and
claude
committed
aebfa2e

Commits on Jul 11, 2026

Split nuPlan manifest entry by split and surface dataset sizes

authored and
Eugene Vinitsky
committed
cd60ac9
Lead nuPlan doc with the default pre-converted bin fetch

authored and
Eugene Vinitsky
committed
a3186a7
Point manifest at pufferdrive-bins prod prefixes; default to mini

authored and
Eugene Vinitsky
committed
0d073bf
Point disabled nuPlan eval placeholders at the default fetch path

authored and
Eugene Vinitsky
committed
57fc612
Merge remote-tracking branch 'origin/ev/nuplan-data-docs' into ev/s3-data-fetch

Eugene Vinitsky
authored and
Eugene Vinitsky
committed
beb22b9
Mark nuPlan datasets public; document the anonymous fetch path

authored and
Eugene Vinitsky
committed
4cad72d
Merge remote-tracking branch 'origin/fix/drive-ini-remove-hardcoded-scratch-paths' into ev/s3-data-fetch

Eugene Vinitsky
authored and
Eugene Vinitsky
committed
5dc2629
Fail fast on missing map_dir with the fetch command in the error

authored and
Eugene Vinitsky
committed
0984d30
Apply ruff format to fetch_data.py

authored and
Eugene Vinitsky
committed
d661b69
Fetch the mini sets by default; post-review fixes and doc trims

authored and
Eugene Vinitsky
committed
ba57514
Install awscli with the package so fetch_data.py has no extra setup

authored and
Eugene Vinitsky
committed
67ccfd0
Trim data docs

authored and
Eugene Vinitsky
committed
141f69c
validation_replay renders via obs_html; drop behaviors block comment

authored and
Eugene Vinitsky
committed
462b506
Drop map_dir error-message test

authored and
Eugene Vinitsky
committed
113d77c
Drop the PUFFERDRIVE_DATA_ROOT env var; --data-root is the only override

authored and
Eugene Vinitsky
committed
b81ce95

Commits on Jul 24, 2026

Merge remote-tracking branch 'origin/3.0' into ev/s3-data-fetch

Eugene Vinitsky
authored and
Eugene Vinitsky
committed
661f238
Address review: doc fixes, drop behaviors evals, sync puffer_drive.yaml

authored and
Eugene Vinitsky
committed
8e01d7c
Trim fetch_data.py docstring

authored and
Eugene Vinitsky
committed
7cc436c
Drop redistribution-licensing section from data_storage.md

authored and
Eugene Vinitsky
committed
aa31d47
Drop env-var comment from the 123Drive convert example

authored and
Eugene Vinitsky
committed
3f4e373
Drop map_dir error-message test

authored and
Eugene Vinitsky
committed
747d74c
Trim README WOMD bullet and setup.py awscli comment

authored and
Eugene Vinitsky
committed
71fee5d
Drop fetch-hint comments from validation_replay config

authored and
Eugene Vinitsky
committed
39c09a3