[Feature] MoE模型里稠密层和专家层zero和并行的解耦 #325

Open

Assignees

Labels

opened

on Sep 11, 2024

Describe the feature

MoE模型里稠密层和专家层zero和并行的解耦

Will you implement it?

I would like to implement this feature and create a PR!

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

blankde

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests