test: add FSDP2 distributed coverage for AsyncGRPOTrainer

[behroozazarkhalili]

What

Adds a 2-process FSDP2 functional test for the experimental AsyncGRPOTrainer, which previously had no distributed-test coverage (tests/experimental/test_async_grpo_trainer.py only ran single-process).

test_train_fsdp2 launches a companion worker via accelerate launch under a 2-process FSDP2 config and asserts the trainer trains end-to-end on FSDP2-sharded parameters: training steps run, the loss is finite, and parameters update. It uses an in-process stub rollout worker (no vLLM server / NCCL weight transfer), so the only distributed surface exercised is the FSDP2 parameter lifecycle. The test is guarded by @require_torch_multi_accelerator, so it skips automatically when fewer than 2 accelerators are available.

Why

AsyncGRPOTrainer runs under FSDP2 in practice but had no test confirming it trains under a real 2-process FSDP2 group. This closes that gap with a lightweight functional smoke that mirrors the existing _openreward_echo_env.py companion-script pattern.

Test plan

Verified on 2x H100:

pytest tests/experimental/test_async_grpo_trainer.py::TestAsyncGRPOTrainer::test_train_fsdp2
# 1 passed in 76.10s

Skips cleanly on single-GPU / CPU. ruff check and ruff format pass.

---

Note

Low Risk
Test-only changes with no production trainer or runtime behavior modifications.

Overview
Adds 2-process FSDP2 functional coverage for experimental AsyncGRPOTrainer, which previously only had single-process training tests.

A new test_train_fsdp2 (guarded by @require_torch_multi_accelerator) spawns _async_grpo_fsdp2_worker.py via accelerate launch and a dedicated fsdp2_reshard.yaml (FSDP v2, fsdp_reshard_after_forward: true). The worker runs a short AsyncGRPOTrainer job with an in-process stub rollout (no vLLM / NCCL weight sync), then emits a single ASYNC_GRPO_FSDP2_RESULT JSON line; pytest asserts steps ran, finite train loss, and parameters changed after materializing FSDP2 DTensors for comparison. PYTHONPATH is pinned to the repo root so the subprocess exercises the working tree, not an installed trl package.

^{Reviewed by Cursor Bugbot for commit d6d198d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.