feat: Add /v1/capacity endpoint, 429 handler, and optional admission gate#583
Open
prein wants to merge 2 commits into
Open
feat: Add /v1/capacity endpoint, 429 handler, and optional admission gate#583prein wants to merge 2 commits into
/v1/capacity endpoint, 429 handler, and optional admission gate#583prein wants to merge 2 commits into
Conversation
Signed-off-by: Pawel Rein <pawel.rein@prezi.com>
Contributor
|
✅ DCO Check Passed Thanks @prein, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Signed-off-by: Paweł Rein <pawel.rein@prezi.com>
Contributor
Author
|
We are running it in our preprod environment and it works as expected. It would be amazing if you could find time to review it and hopefully merge and release so we don't have to maintain it in a fork. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on docling-project/docling-jobkit#126 (
SystemCapacity+get_capacity()).See #581 for full problem descriptio
Summary
GET /v1/capacity- returns system-level capacity snapshot (queue depth, active jobs, worker count); cached with configurable TTL (default 2s) to avoid per-request Redis scansQueueLimitExceededError→ 429 - fixes unhandled 500 when Ray queue limit rejection is enabled; returns capacity snapshot in response bodyadmission_max_queue_sizesetting; when set, rejects submissions with 429 beforeenqueue()if queue is fullIssue resolved by this Pull Request:
Resolves #581