Data onboarding: remove hardcoded paths, S3 dataset fetch, nuPlan docs

[eugenevinitsky]

What

Data onboarding: datasets move to S3 with a one-command fetch, and the hardcoded cluster paths in the default config are gone.

• data_utils/fetch_data.py + data_utils/datasets.yaml: registry of shared S3 datasets; a bare run fetches the ~10 GB nuPlan mini train/val sets (public bucket, no AWS account needed). awscli added to install deps.
• docs/data_storage.md (S3 layout, fetching, adding datasets) and docs/nuplan_data.md (full download → py123d → 123Drive conversion pipeline), plus a README data section.
• Drive fails fast on a missing map_dir; when the path's basename is a registered dataset, the error names the exact fetch command (covered by tests/unit_tests/test_map_dir_missing.py).
• validation_replay now ships disabled with env.map_dir preset to data/nuplan_mini_val — where the default fetch lands — in both puffer_drive.yaml (the config Hydra loads) and drive.ini.
• All eval.behaviors_* evaluators (defaults, full_dir, and the 12 scene-category sections) are deleted along with their cluster-config overrides — their labelled-category data isn't onboardable to S3 and they were unused.

Why

Getting nuPlan data previously meant copying paths from someone's cluster home directory; the default config pointed at /scratch/ev2237/... and silently broke for everyone else. Now the data story is: clone, python data_utils/fetch_data.py, train — and any config that needs data that isn't fetched says exactly how to get it.

Notes

• All review corrections from the doc walkthrough are applied: env vars (NUPLAN_DATA_ROOT, NUPLAN_MAPS_ROOT, PY123D_DATA_ROOT) exported before the commands that need them, py123d-conversion dataset=nuplan-mini syntax, --datasets nuplan-mini on the 123Drive convert, and the converter's env-var map resolution documented.
• Doc CLI examples use Hydra override syntax (env.map_dir=...); the old --env.map-dir flag form is rejected by load_config since the Hydra switch.
• Branch is merged up to current 3.0; 83 unit tests pass.

🤖 Generated with Claude Code

[riccardosavorgnan]

This returns an error, I think the correct syntax to override is:
py123d-conversion dataset=nuplan-mini

Error log:

(pufferdata_2) (base) ricky@rickynyuserver:~/pufferdata_2$ py123d-conversion datasets=["nuplan-mini"]
/home/ricky/miniconda3/lib/python3.13/site-packages/requests/__init__.py:86: RequestsDependencyWarning: Unable to find acceptable character detection dependency (chardet or charset_normalizer).
  warnings.warn(
Could not override 'datasets'.
To append to your config use +datasets=[nuplan-mini]
Key 'datasets' is not in struct
    full_key: datasets
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

[riccardosavorgnan]

this does not find the logs correctly.

After processing the maps according to the previous instructions, py123d outputs 2 folders ./None/maps, ./None/logs in the directory where you run the command. When I then run the command pointing to that folder I get the following error:

(pufferdata_2) (base) ricky@rickynyuserver:~/pufferdata_2/123Drive$ uv run convert --preset nuplan --py123d_path ~/pufferdata_2/None/ --output ./nuplan_bins
warning: VIRTUAL_ENV=/home/ricky/pufferdata_2/.venv does not match the project environment path .venv and will be ignored; use --active to target the active environment instead
INFO bin_factory: 123Drive: /home/ricky/pufferdata_2/None/ -> ./nuplan_bins
INFO bin_factory: Filters - datasets: ['nuplan'], split_types: None, split_names: None, log_names: None, duration_s: 20, map_only: False
INFO bin_factory: No scenarios to process.

[riccardosavorgnan]

From claude: the error appears because the nuplan-mini dataset has a different prefix for the scenes so the command scans the folder and doesn't find any. The fix is to pass the dataset name (nuplan-mini) as a flag in the command.

uv run convert --preset nuplan --datasets nuplan-mini --py123d_path ~/pufferdata_2/None/ --output ./nuplan_bins

[riccardosavorgnan]

This command also requires to set
export PY123D_DATA_ROOT=/path/to/outputofpy123d/
otherwise the conversion fails.
The reason is that internally the call to the maps data is performed via this path and not via the flag --py123d_path