feat(eval): build SWE-bench-style eval instances from PR-linked captures (ALL-2451)

[simonrosenberg]

Why

When an OpenHands Cloud conversation produces a pull request, we'd like to replay it as an evaluation instance. An eval needs the initial repo state the agent started from (repo + base_commit), the task (problem_statement), and the resulting change (patch). This is the data-shaping kernel of ALL-2451 — it turns a PR-linked capture into a dataset row in the exact shape the SWE-bench harness already consumes, so captured conversations become replayable evals. It pairs with the workspace-archive work (software-agent-sdk#3867, runtime-api#623, OpenHands#14953) that captures the final git delta.

What

• benchmarks/utils/pr_snapshot.py:
  • PRSnapshot — a PR-linked capture (repo, pr_number, base_commit, title/body, head_commit, optional patch).
  • build_swebench_instance() — emits a row with the fields swebench/run_infer.py reads (instance_id, repo, base_commit, problem_statement) plus patch and provenance. JSON-serialisable; instance_id follows the owner__repo-<n> convention.
  • pr_snapshot_from_github_json() — accepts both the gh pr view --json (GraphQL) and REST PR shapes; derives owner/repo from the URL when absent; supports an explicit base_commit override (a PR's base ref can advance after the conversation started).
  • write_instances_jsonl() — writes rows to a .jsonl dataset.
• benchmarks/scripts/pr_to_eval_instance.py — CLI the PR-merge capture job calls to emit/append one instance from a PR payload.

Scope / follow-up

This is the deterministic kernel. The PR-merge trigger + clone/delta-storage Kubernetes job (ALL-2451's "trigger a job when a PR goes MERGED/CLOSED; update the delta, not the whole repo") is the remaining infra piece and needs a deploy/automation home + product decisions on the dataset destination — out of scope for this PR. This is what that job calls per instance.

How to Test

uv run pytest tests/test_pr_snapshot.py -v

Covers field mapping (with/without body), both GitHub JSON shapes, owner/repo derived from the URL, the explicit base-commit override, and a JSONL roundtrip through get_dataset. ruff check/format clean.

[simonrosenberg]

Closing — re-homing ALL-2451. benchmarks is the eval harness (dataset consumer); ALL-2451's core is a capture pipeline (PR-merge trigger → clone/git-delta → object storage), which belongs in a data-pipeline service (deploy/data_platform), alongside ALL-2346. Rebuilding there with the eval-instance format folded in. The PR→instance shaping logic from this PR will be reused.