Skip to content

feat(rl): 接入 SWE-Smith Code Agent 与 Qwen3.5/3.8 文本模型 - #1100

Open
haiyangxin wants to merge 1 commit into
mindspore-ai:masterfrom
haiyangxin:master-dev
Open

feat(rl): 接入 SWE-Smith Code Agent 与 Qwen3.5/3.8 文本模型#1100
haiyangxin wants to merge 1 commit into
mindspore-ai:masterfrom
haiyangxin:master-dev

Conversation

@haiyangxin

@haiyangxin haiyangxin commented Oct 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind feature

What does this PR do / why do we need it:

将 SWE-Smith 仓库修复任务接入公共 AgentProgram / SyncTrainer，支持 Qwen3.5-9B 与 Qwen3.8-27B 文本模型的 Code Agent RL 流程。

  • 数据与任务：子集/全量转换、按任务准备镜像、隔离候选与 grader、冻结补丁和独立评分，支持本地或 SSH 远端 Docker。
  • 模型与训练：动作位置概率、分块词表投影、逐调用 GRPO、共卡 native vLLM、流式 full_gather、权重发布和完整恢复。
  • 命名与兼容：整理 RL 内部标识符，保留现有配置值、插件入口、模型注册名称和共享模型 API。
  • HSDP：参数加载后刷新 storage 使用 no_grad，保持参数身份并释放旧 shard，附对应回归测试。
  • 整理模型配方、使用说明与开发文档，拆分复杂校验与执行步骤。
  • 清理和异步传输边界保留完整异常保护并注明用途，避免覆盖主错误或丢失故障归因。

Which issue(s) this PR fixes:

Fixes #1099

Test Plan and Test result：What scenarios were tested, and what were the verification results（Function, performance, reliability, etc.）：

  • 完整 RL、Qwen3.5 概率适配、HSDP 修复及 ST 支持合同：869 项 CPU 测试通过。
  • 9B 短预算 RL、权重发布与完整恢复通过；受控奖励下的非零更新与恢复通过。
  • 三仓库九任务的基线/可信修复对照通过；选定任务多次推理获得真实混合奖励。
  • 目录检查与 275 条文档相对链接检查通过。

Self-checklist:（请自检，在[ ]内打上x，我们将检视你的完成情况，否则会导致pr无法合入）

  • 设计：PR对应的方案是否已经经过Maintainer评审，方案检视意见是否均已答复并完成方案修改
  • 测试：PR中的代码是否已有UT/ST测试用例进行充分的覆盖，新增测试用例是否随本PR一并上库或已经上库
  • 验证：PR描述信息中是否已包含对该PR对应的Feature、Refactor、Bugfix的预期目标达成情况的详细验证结果描述
  • 接口：是否涉及对外接口变更，相应变更已得到接口评审组织的通过，API对应的注释信息已经刷新正确
  • 文档：是否涉及官网文档修改，如果涉及请及时提交资料到Doc仓

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/checkp-pr

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/check-pr

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown

🔵 The pipeline #575 is running. Please wait a moment... (Link)

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ❌ FAILURE 575
Check_Lizard FAILURE 575
Check_Pylint FAILURE 575

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown

@haiyangxin, your gate pipeline failed at the code check stage. You can configure some code check tools locally following the (pre-push quick guide), to enable automatic checking when you push your code every time. Tools included are: Check_CMakelint, Check_Codespell, Check_Cpplint, Check_Lizard, Check_Pylint, Check_ClangFormat, Check_ShellCheck.

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/check-pr

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown

🔵 The pipeline #576 is running. Please wait a moment... (Link)

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details(Full/Incremental) Detail
Coverage Line Coverage ❌ FAILURE 19%/100% Link

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ❌ FAILURE 576
Check_Lizard FAILURE 575
Check_Pylint FAILURE 575
UT_TEST FAILURE 576
coverage FAILURE 576

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown

@haiyangxin, your gate pipeline failed at the code check stage. You can configure some code check tools locally following the (pre-push quick guide), to enable automatic checking when you push your code every time. Tools included are: Check_CMakelint, Check_Codespell, Check_Cpplint, Check_Lizard, Check_Pylint, Check_ClangFormat, Check_ShellCheck.

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/check-pr

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown

🔵 The pipeline #577 is running. Please wait a moment... (Link)

@haiyangxin
feat(rl): integrate SWE-Smith agents and Qwen3.5-family models
abbfa7c 
接入仓库型 Code Agent、Qwen3.5-9B 与 Qwen3.8-27B 的 RL 流程，
整理模型配方、功能验证、使用说明及当前限制。

整改 RL 内部变量、辅助函数和私有状态命名；保留现有 hyper/native
配置值、插件入口、模型注册名称及 RL 外共享 API，保持运行环境兼容。

验证：869 项 CPU 测试通过。
@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details(Full/Incremental) Detail
Coverage Line Coverage ❌ FAILURE 65%/100% Link

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ❌ FAILURE 577
Check_Lizard FAILURE 575
Check_Pylint FAILURE 575
UT_TEST FAILURE 577
UT_TEST FAILURE 576
coverage FAILURE 577
coverage FAILURE 576

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown

@haiyangxin, your gate pipeline failed at the code check stage. You can configure some code check tools locally following the (pre-push quick guide), to enable automatic checking when you push your code every time. Tools included are: Check_CMakelint, Check_Codespell, Check_Cpplint, Check_Lizard, Check_Pylint, Check_ClangFormat, Check_ShellCheck.

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/check-pr

@haiyangxin

haiyangxin commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown

🔵 The pipeline #578 is running. Please wait a moment... (Link)

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details(Full/Incremental) Detail
Coverage Line Coverage ✅ SUCCESS 65%/100% Link

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Platform Type Module Excution Time Time Limit
_torch - - -
- activation_memory 112.47062s 120.0s
- shard 148.809s 300.0s
- pipeline_parallel 490.351s 600.0s
- optimizer 205.082s 300.0s
- swap_optimizer 70.716s 120.0s
- clip_grad 69.421s 120.0s
- multicore 70.63187s 120.0s
- context_parallel 66.286s 120.0s
- fully_shard 254.148s 600.0s
- distributed_checkpoint 66.812s 120.0s
- tensor_parallel 193.38s 300.0s
- accuracy 66.778s 120.0s
- expert_parallel 62.499s 120.0s
- init_weights 61.84s 118.75s
- dtensor 62.08s 120.0s
- process_group 60.802s 90.0s
- llama3_examples 60.174s 120.0s

@mindspore-ai-bot

mindspore-ai-bot commented Oct 11, 2026

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ✅ SUCCESS 578

liuchongming74
liuchongming74 approved these changes Oct 12, 2026
View reviewed changes

@liuchongming74 liuchongming74 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@suteng suteng suteng approved these changes
@liuchongming74 liuchongming74 liuchongming74 approved these changes
@yangzhenzhang02 yangzhenzhang02 yangzhenzhang02 approved these changes
@yaoyifanx yaoyifanx Awaiting requested review from yaoyifanx yaoyifanx is a code owner

Assignees

No one assigned

Labels

ci-pipeline-passed pr-check-pass

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature]: 接入 SWE-Smith Code Agent 与 Qwen3.5/3.8 RL 文本模型

5 participants

@haiyangxin @mindspore-ai-bot @suteng @liuchongming74 @yangzhenzhang02