Problem
Bulk imports created through /ui/bulk-import never release their per-tenant concurrency slot. After 4 imports (default HFS_BULK_SUBMIT_MAX_CONCURRENT_PER_TENANT), every further kick-off is rejected with:
429 — too many concurrent submissions for this tenant (max 4)
Reproduced on a fresh SQLite server: four UI imports whose manifests all reached completed (20,000/20,000 entries each) still sit as in-progress rows in bulk_submissions, and a fifth kick-off 429s.
Why
A recipient-side submission only turns terminal when the submitter sends a kick-off with submissionStatus=completed (or the submission is aborted). The UI sends in-progress with each manifest kick-off and never follows up: when its status poll sees the 200 manifest it marks its own document completed ("processing finished cleanly; submission completed") but tells the recipient nothing. The manifests are terminal, the submission row is not, and count_active_submissions keeps counting it forever.
This matches the STU4 model — the Data Provider owns submission completion, and keeping the submission open is what makes replacesManifestUrl (#531) possible — but the UI currently has no path that ever closes it (only Abort → stopped).
Options
- Auto-complete: when the UI's poll lands the 200 status manifest, also send the
submissionStatus=completed kick-off. Simple, matches what the log already claims; costs the ability to replace a manifest after completion (arguably correct — replacing after "completed" should be a new submission).
- Explicit control: a "Mark completed" action next to Abort, plus surfacing open-slot pressure. Preserves replace-after-finish; relies on the user knowing to click it.
- Either of the above plus the recipient auto-finalizing submissions whose manifests are all terminal after some grace period, as belt-and-braces against any submitter that walks away.
Workaround meanwhile: abort stuck submissions, send the completed kick-off manually, or raise HFS_BULK_SUBMIT_MAX_CONCURRENT_PER_TENANT.
Found while testing #848/#849 (the byte-based progress work): the probe submissions themselves burned the slots.
Problem
Bulk imports created through
/ui/bulk-importnever release their per-tenant concurrency slot. After 4 imports (defaultHFS_BULK_SUBMIT_MAX_CONCURRENT_PER_TENANT), every further kick-off is rejected with:Reproduced on a fresh SQLite server: four UI imports whose manifests all reached
completed(20,000/20,000 entries each) still sit asin-progressrows inbulk_submissions, and a fifth kick-off 429s.Why
A recipient-side submission only turns terminal when the submitter sends a kick-off with
submissionStatus=completed(or the submission is aborted). The UI sendsin-progresswith each manifest kick-off and never follows up: when its status poll sees the 200 manifest it marks its own document completed ("processing finished cleanly; submission completed") but tells the recipient nothing. The manifests are terminal, the submission row is not, andcount_active_submissionskeeps counting it forever.This matches the STU4 model — the Data Provider owns submission completion, and keeping the submission open is what makes
replacesManifestUrl(#531) possible — but the UI currently has no path that ever closes it (only Abort →stopped).Options
submissionStatus=completedkick-off. Simple, matches what the log already claims; costs the ability to replace a manifest after completion (arguably correct — replacing after "completed" should be a new submission).Workaround meanwhile: abort stuck submissions, send the completed kick-off manually, or raise
HFS_BULK_SUBMIT_MAX_CONCURRENT_PER_TENANT.Found while testing #848/#849 (the byte-based progress work): the probe submissions themselves burned the slots.