AsyncGRPOTrainer: add ProcessorMixin handling

[rycerzes]

What does this PR do?

Adds ProcessorMixin support to AsyncGRPOTrainer, matching the existing behavior in GRPOTrainer.

Currently, AsyncGRPOTrainer only uses AutoTokenizer when no processing_class is provided, and only accepts PreTrainedTokenizerBase instances. This fails for models like Qwen3.5 that use a processor (e.g., Qwen3_5ForConditionalGeneration returns a ProcessorMixin from AutoProcessor).

This PR:

• Uses AutoProcessor.from_pretrained() instead of AutoTokenizer.from_pretrained() when no processing_class is provided
• Accepts both PreTrainedTokenizerBase and ProcessorMixin as processing_class
• Extracts the tokenizer from ProcessorMixin instances (processing_class.tokenizer)
• Passes the extracted tokenizer (not the processor) to super().__init__() and AsyncRolloutWorker
• Normalizes pad token on the tokenizer (pad_token ← eos_token when not set)

Changes:

• async_grpo_trainer.py: update imports (AutoProcessor, ProcessorMixin), update processing_class type hint and docstring, add processor/tokenizer extraction logic

Tests:

• test_processor_mixin_handling: passes an AutoProcessor instance as processing_class and verifies the trainer extracts a PreTrainedTokenizerBase
• test_auto_processor_when_none: omits processing_class entirely and verifies the trainer auto-loads via AutoProcessor with a valid pad token

Closes part of #5831

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline, Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

AI writing disclosure

• No AI usage: the PR was written entirely by a human.
• AI-assisted: some parts were suggested or improved by AI, but the PR was written and reviewed by a human.
• AI-generated: the PR was mostly or fully generated by an AI tool.

Who can review?

@qgallouedec

---

Note

Medium Risk
Touches core trainer initialization, model loading, and tokenization paths used by async GRPO training; changes are scoped and mirrored from GRPOTrainer with new tests.

Overview
AsyncGRPOTrainer now aligns with GRPOTrainer for multimodal and model-loading behavior. When no processing_class is passed, it loads AutoProcessor (left truncation/padding) instead of AutoTokenizer, accepts ProcessorMixin or PreTrainedTokenizerBase, normalizes pad_token on the inner tokenizer, and passes that tokenizer to the rollout worker and data collator while still handing the full processor to the base trainer.

Model weights are loaded via create_model_from_path with optional AsyncGRPOConfig.model_init_kwargs (e.g. dtype), with device_map=None forced for FSDP2.

Tests add model_init_kwargs coverage, extend the rollout stub for processor-only prompt data, and add a gated VLM training test that checks non-visual weights update while visual weights stay fixed on text-only data.

^{Reviewed by Cursor Bugbot for commit af4f768. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 90afff0. Configure here.}

[qgallouedec]

at this point I don't know if it's very useful to integrate processor, a the asynchronous grpo trainer doesn't support vlm yet.

[rycerzes]

at this point I don't know if it's very useful to integrate processor, a the asynchronous grpo trainer doesn't support vlm yet.

Yes I agree, this isn’t for full VLM support. The scope is text-only training for processor-backed models like Qwen3.5

Updated to stack on #5893, tests passing :)