🚀 [2026/06/18] Recent Updates Summary for ROLL Project

[PanAndy]

大家好！感谢大家对ROLL的关注。ROLL发布了v0.3.0版本，新增Video RLVR、AgentRunner 2.0、MTP训练、Router Replay、Multi-Teacher OPD等重要特性；新增OpenTelemetry可观测性支持；强化mcore_adapter能力；扩展NPU/AMD硬件适配。以下是近期更新的一些梳理，我们将持续对ROLL进行迭代更新，欢迎加入ROLL的社区。

🚀 亮点

• 新增 Video/Audio RLVR 训练支持（Video-R1 reward）
• 新增 AgentRunner 2.0 抽象，解耦Agent交互逻辑，支持更灵活的多轮Agent场景
• 新增 RemoteBatch 惰性数据传输机制，优化大规模 image/video/long_context logits 跨Worker传输
• 新增 MoE Router Replay (R3)，MoE模型训练时复用rollout阶段的路由决策
• 支持 Qwen3.5/3.6 MTP (Multi-Token Prediction) SFT/RL 训练
• 新增 OpenTelemetry 分布式追踪，提供端到端可观测性

🚀 主要新特性

Pipeline

• 新增 Video/Audio RLVR Pipeline，支持视频、音频理解场景的强化学习训练(Qwen3Omni系列模型)
• 新增 Multi-Teacher On-Policy Distillation 支持，[文档](docs_roll/i18n/zh-Hans/docusaurus-plugin-content-docs/current/User Guides/Pipeline/on_policy_distill_pipeline_start.md)
• 新增 LLM-as-Judge Server 模式，支持独立部署 judge 服务，示例配置

Agent Native 2.0

• 新增 AgentRunner 抽象，解耦"Agent如何与环境交互"与"训练样本构造"，[设计文档](docs_roll/i18n/zh-Hans/docusaurus-plugin-content-docs/current/User Guides/Agentic/agent_runner.md)
• 新增 ProxyEnvManager / MessageTracker，支持更复杂的Agent交互模式，[设计文档](docs_roll/i18n/zh-Hans/docusaurus-plugin-content-docs/current/User Guides/Agentic/prefix_aggregation.md)
• 新增 Atropos 环境集成，示例配置
• 新增 OpenReward 环境集成，示例配置

mcore_adapter

• 新增 MTP (Multi-Token Prediction) 训练支持（standalone/joint两种模式），[使用文档](docs_roll/i18n/zh-Hans/docusaurus-plugin-content-docs/current/User Guides/Advanced Features/mtp_training.md)
• 新增 Router Replay (R3)，MoE训练时复用rollout路由决策，减少重计算开销，[使用文档](docs_roll/docs/User Guides/Advanced Features/router_replay.md)
• 新增 Fused Entropy CE kernel，TP=1场景下加速交叉熵计算
• 新增 PP Stage Compile Warmup，Pipeline并行编译预热
• Qwen3.5/3.6系列 VLM sequence packing 优化

RemoteBatch 传输优化

• 新增惰性数据传输后端，支持 image/video/long_context 场景下大规模数据高效传输，[使用文档](docs_roll/i18n/zh-Hans/docusaurus-plugin-content-docs/current/User Guides/Advanced Features/remote_batch_transfer.md)
• 基于TransferQueue优化 Ray Worker 间存储管理

Observability

• 新增 OpenTelemetry 集成，支持分布式追踪，[使用文档](docs_roll/i18n/zh-Hans/docusaurus-plugin-content-docs/current/User Guides/Advanced Features/opentelemetry_tracing.md)
• 新增 OTEL Receiver，pipeline各阶段端到端tracing

FSDP2

• 新增 Qwen3 MoE patch，支持 MoE 模型 FSDP2 训练
• LoRA模型支持优化
• FSDP2 / EP并行支持

Docker

• 新增 NPU A2/A3 Docker 镜像
• 新增 AMD torch2.8.0/torch2.10 Docker 镜像

Hardware

• NPU：新增 A2/A3 适配，修复 FSDP2 相关问题，新增 Ascend 全流程文档
• AMD：新增 torch2.10 支持，ROCm参数同步优化

Models

• 支持 Qwen3.5 Dense (27B) / MoE (35B-A3, 122B-A10, 397A-17) 系列模型
• 提供 Megatron + FSDP2 多种规格配置示例

Performance 优化

• do_checkpoint pin_memory 优化
• GC 优化
• low-memory checkpoint convert

Bug Fix

• fix sglang & vllm 偶现 port conflict
• fix reward worker metrics 透出
• fix vllm GDN attention mixed decode/spec-decode crash（vllm < 0.17.2）

Deprecated

• DeepSpeed Strategy（third_party代码已移除）
• Wan RewardRL（生成模型的RL训练重构中）

TODOs

• Multi Agent 支持
• Full vocab version Multi-Teacher OPD

[PanAndy]

Hello everyone! Thank you for your interest in ROLL. We are releasing ROLL v0.3.0, featuring major new capabilities including Video RLVR, AgentRunner 2.0, MTP Training, Router Replay, Multi-Teacher OPD, and more. This release also introduces OpenTelemetry observability support, enhanced mcore_adapter capabilities, and expanded NPU/AMD hardware support. Below is a summary of the recent updates. We will continue iterating on ROLL — welcome to join the ROLL community.

Highlights

• Video/Audio RLVR training support (Video-R1 reward)
• AgentRunner 2.0 abstraction — decouples agent interaction logic for more flexible multi-turn agent scenarios
• RemoteBatch lazy data transfer mechanism — optimizes large-scale image/video/long_context logits transfer across workers
• MoE Router Replay (R3) — reuses rollout-phase routing decisions during MoE model training
• Qwen3.5/3.6 MTP (Multi-Token Prediction) SFT/RL training support
• OpenTelemetry distributed tracing for end-to-end observability

Major New Features

Pipeline

• Video/Audio RLVR Pipeline — supports reinforcement learning training for video and audio understanding scenarios (Qwen3Omni series models)
• Multi-Teacher On-Policy Distillation support, [documentation](docs_roll/docs/User Guides/Pipeline/on_policy_distill_pipeline_start.md)
• LLM-as-Judge Server mode — supports standalone judge service deployment, example config

Agent Native 2.0

• AgentRunner abstraction — decouples "how agents interact with environments" from "training sample construction", [design doc](docs_roll/docs/User Guides/Agentic/agent_runner.md)
• ProxyEnvManager / MessageTracker — supports more complex agent interaction patterns
• Atropos environment integration, example config
• OpenReward environment integration, example config

mcore_adapter

• MTP (Multi-Token Prediction) training support (standalone/joint modes), [documentation](docs_roll/docs/User Guides/Advanced Features/mtp_training.md)
• Router Replay (R3) — reuses rollout routing decisions during MoE training to reduce recomputation overhead, [documentation](docs_roll/docs/User Guides/Advanced Features/router_replay.md)
• Fused Entropy CE kernel — accelerates cross-entropy computation in TP=1 scenarios
• PP Stage Compile Warmup — pipeline parallelism compile warmup
• Qwen3.5/3.6 VLM sequence packing optimization

RemoteBatch Transfer Optimization

• Lazy data transfer backend — enables efficient large-scale data transfer for image/video/long_context scenarios, [documentation](docs_roll/docs/User Guides/Advanced Features/remote_batch_transfer.md)
• Storage management optimization between Ray Workers based on TransferQueue

Observability

• OpenTelemetry integration with distributed tracing support, [documentation](docs_roll/docs/User Guides/Advanced Features/opentelemetry_tracing.md)
• OTEL Receiver — end-to-end tracing across all pipeline stages

FSDP2

• Qwen3 MoE patch — supports MoE model FSDP2 training
• LoRA model support optimization
• FSDP2 / EP parallelism support

Docker

• NPU A2/A3 Docker images
• AMD torch2.8.0/torch2.10 Docker images

Hardware

• NPU: A2/A3 support, FSDP2-related fixes, full Ascend workflow documentation
• AMD: torch2.10 support, ROCm parameter synchronization optimization

Models

• Qwen3.5 Dense (27B) / MoE (35B-A3, 122B-A10, 397A-17) series models
• Megatron + FSDP2 multi-scale configuration examples

Performance Optimization

• do_checkpoint pin_memory optimization
• GC optimization
• Low-memory checkpoint conversion

Bug Fixes

• Fix sglang & vllm occasional port conflict
• Fix reward worker metrics exposure
• Fix vllm GDN attention mixed decode/spec-decode crash (vllm < 0.17.2)

Deprecated

• DeepSpeed Strategy (third_party code removed)
• Wan RewardRL (RL training for generative models under refactoring)

TODOs

• Multi-Agent support
• Full vocab version Multi-Teacher OPD