Skip to content

[New Post] From Load Balancing to Self-Healing: Progressive KV Cache Architecture for DeepSeek-V3.2 on AMD Instinct#381

Open
github-actions[bot] wants to merge 2 commits into
releasefrom
mpashkovskii-380
Open

[New Post] From Load Balancing to Self-Healing: Progressive KV Cache Architecture for DeepSeek-V3.2 on AMD Instinct#381
github-actions[bot] wants to merge 2 commits into
releasefrom
mpashkovskii-380

Conversation

@github-actions

@github-actions github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

📝 Blog Post Pull Request

Hello @mpashkovskii,

This is the pull request for your blog post and closes #380. The automated system has created the initial blog structure for you.


LEGAL REVIEW - REQUIRED BEFORE PUBLICATION

Caution

BLOCKING: Your blog WILL NOT be published until ALL items below are checked.
A CI check enforces this requirement. The PR cannot be merged with an incomplete legal review.

  • Legal self review completed - AMD Employees: Complete traffic lights review
  • Performance claims cleared - If the blog contains ANY performance benchmarks, comparisons, or claims, you MUST obtain legal clearance before publication.
    • Any claim that references performance or improvements should either be:
      - Backed up with measurements and a % improvement, with the required legal claim/footnote.
      - Highlight what we enabled and what it was intended to do without making claims about what the improvement is relative to (unless there is a measured performance claim).
  • License check - Verified all licenses for code, images, and third-party content are correct and properly attributed.

📋 Next Steps

Please continue working on the blog via the pull request. You can:

  • Review the metadata carefully for any mistakes or adjustments needed
  • Edit the content in the Markdown files to complete your blog post
  • Request reviews from team members when ready
  • Address any feedback before final publication

✅ Content Checklist

  • Content follows ROCm Tech Blogs Style Guide
  • Post aligns with Diataxis framework guidelines
  • All images/diagrams are properly attributed and have alt text
  • Code samples (if any) are tested and properly formatted
  • Technical accuracy verified
  • No confidential information included
  • Reviewed recently published blog posts and existing Pull Requests to confirm there are no published or in-flight posts on this same topic

✅ Signoffs and Approvals

Please complete the steps below to the best of your ability. This section must be completed before publishing.

  • Technical reviewer approval
  • Editorial team approval
  • Thumbnail: Thumbnail image for blog post included in PR
  • Post description: Provide a 2-3 line summary that grabs the reader's attention:

✅ Blog Author/Team Signoffs

  • Post-review changes: Confirm that changes from technical review and editorial feedback have been made
  • All sections above are completed
  • Content is in final draft state
  • Images/assets are optimized and properly placed
  • Linter checks are passing

Generated By: GitHub Issue #380
Creation Date: Jun 02, 2026
Corresponding Jira Ticket: No response

…Architecture for DeepSeek-V3.2 on AMD Instinct
- Introduced chaos-monkey.yaml to implement a Chaos Monkey for killing the oldest vLLM worker pod to test resilience.
- Added rdma-infra.yaml to deploy Mellanox RDMA shared device plugin as a DaemonSet for RDMA infrastructure.
- Created stage1-baseline.yaml for the initial setup of vLLM Router with consistent hashing and two vLLM instances.
- Developed stage2-lmcache-cpu.yaml to implement LMCache CPU offloading with increased memory limits and environment variables.
- Implemented stage3-distributed.yaml for a distributed KV cache using LMCache and Mooncake, including a dedicated CPU node for Mooncake components.
@mpashkovskii

Copy link
Copy Markdown

It is still WIP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[New Post] From Load Balancing to Self-Healing: Progressive KV Cache Architecture for DeepSeek-V3.2 on AMD Instinct

1 participant