Skip to content

ERROR:train sparse model with FDSP2 #821

@Vieeo

Description

@Vieeo

Bug:

How to solve it?

masked_weight = weight * mod._weight_mask
RuntimeError: aten.mul.Tensor: got mixed torch.Tensor and DTensor, need to convert all torch.Tensor to DTensor before calling distributed operators!

system info:

torch 2.9.0+cu12.6
modelopt 0.41

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions