Are community contributions welcome? #51

Activity

1. hijkzzz commented on Jul 30, 2026

   

   hijkzzz

   on Jul 30, 2026

   Contributor

   More actions

   Sure! Welcome to contribute.

   ❤️React with ❤️2Reacted by NancyFyong and Yunhao Chen
2. NancyFyong commented on Jul 30, 2026

   

   NancyFyong

   on Jul 30, 2026

   ContributorAuthor

   More actions

   Sure! Welcome to contribute.

   Thanks! I'd like to add LoRA / PEFT support to Molt. Rather than opening a PR directly,
   I'll write an RFC first — the weight-sync design has a couple of reasonable options, and
   I'd prefer to settle that with you up front.

   Why I think this is a small change

   • AutoModel already does the hard part: from_pretrained(..., peft_config=...) injects
     the adapters before sharding, and expert LoRA is EP-aware.
   • peft is already listed in requirements.txt, with nothing currently importing it.

   The part that needs real work is refit. Under LoRA the base weights are frozen, so refit
   has to either:

   • materialize W0 + scale*BA, or
   • switch to native vLLM LoRA.

   Skipping the adapter tensors would push only the unchanged weights and leave the rollout
   engine stuck on the initial policy. Forwarding them as-is fails the flush outright —
   AutoWeightsLoader raises on names it has no parameter for.

   What the RFC will cover

   • Scope — RL + SFT
   • Refit choice — merge on refit vs. native vLLM LoRA (broadcast adapters only)
   • Checkpointing
   • Failure modes — which combinations should fail loudly
   • Test plan

   Open questions (before I write it)

   1. Is LoRA in scope for Molt, or was it deliberately left out?
   2. Where would you like the RFC — a separate issue, or a comment on this thread?
3. NancyFyong commented on Jul 30, 2026

   

   NancyFyong

   on Jul 30, 2026

   ContributorAuthor

   More actions

   Sure! Welcome to contribute.

   Thanks! I'd like to add LoRA / PEFT support to Molt. Rather than opening a PR directly, I'll write an RFC first — the weight-sync design has a couple of reasonable options, and I'd prefer to settle that with you up front.

   Why I think this is a small change

   • AutoModel already does the hard part: from_pretrained(..., peft_config=...) injects
     the adapters before sharding, and expert LoRA is EP-aware.
   • peft is already listed in requirements.txt, with nothing currently importing it.

   The part that needs real work is refit. Under LoRA the base weights are frozen, so refit has to either:

   • materialize W0 + scale*BA, or
   • switch to native vLLM LoRA.

   Skipping the adapter tensors would push only the unchanged weights and leave the rollout engine stuck on the initial policy. Forwarding them as-is fails the flush outright — AutoWeightsLoader raises on names it has no parameter for.

   What the RFC will cover

   • Scope — RL + SFT
   • Refit choice — merge on refit vs. native vLLM LoRA (broadcast adapters only)
   • Checkpointing
   • Failure modes — which combinations should fail loudly
   • Test plan

   Open questions (before I write it)

   1. Is LoRA in scope for Molt, or was it deliberately left out?
   2. Where would you like the RFC — a separate issue, or a comment on this thread?

   @hijkzzz Hi! Would you be able to review this when you have time? Thanks a lot!