[New Post] From Load Balancing to Self-Healing: Progressive KV Cache Architecture for DeepSeek-V3.2 on AMD Instinct#381
Open
github-actions[bot] wants to merge 2 commits into
Open
Conversation
…Architecture for DeepSeek-V3.2 on AMD Instinct
- Introduced chaos-monkey.yaml to implement a Chaos Monkey for killing the oldest vLLM worker pod to test resilience. - Added rdma-infra.yaml to deploy Mellanox RDMA shared device plugin as a DaemonSet for RDMA infrastructure. - Created stage1-baseline.yaml for the initial setup of vLLM Router with consistent hashing and two vLLM instances. - Developed stage2-lmcache-cpu.yaml to implement LMCache CPU offloading with increased memory limits and environment variables. - Implemented stage3-distributed.yaml for a distributed KV cache using LMCache and Mooncake, including a dedicated CPU node for Mooncake components.
|
It is still WIP |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Blog Post Pull Request
Hello @mpashkovskii,
This is the pull request for your blog post and closes #380. The automated system has created the initial blog structure for you.
LEGAL REVIEW - REQUIRED BEFORE PUBLICATION
Caution
BLOCKING: Your blog WILL NOT be published until ALL items below are checked.
A CI check enforces this requirement. The PR cannot be merged with an incomplete legal review.
- Backed up with measurements and a % improvement, with the required legal claim/footnote.
- Highlight what we enabled and what it was intended to do without making claims about what the improvement is relative to (unless there is a measured performance claim).
📋 Next Steps
Please continue working on the blog via the pull request. You can:
✅ Content Checklist
✅ Signoffs and Approvals
Please complete the steps below to the best of your ability. This section must be completed before publishing.
✅ Blog Author/Team Signoffs
Generated By: GitHub Issue #380
Creation Date: Jun 02, 2026
Corresponding Jira Ticket: No response