-
Notifications
You must be signed in to change notification settings - Fork 106
Open
Description
Hi, I was working on the adls labs when I found out that the linearBlockMinifloat has backward which takes arguments same as the forward pass:
mase/src/chop/nn/quantizers/block_minifloat.py
Lines 84 to 92 in 488d49e
| def backward( | |
| ctx, | |
| grad_output: Tensor, | |
| width: int, | |
| exponent_width: int, | |
| exponent_bias_width: int, | |
| block_size: list[int] | int = [16], | |
| skip_first_dim: bool = False, | |
| ): |
Other precision backward passes do not have this arguments. Further, torch when calculating backprop wont pass these arguments, hence causing Missing 3 positional arguments error (Block size has default). This is either a bug, or have not been implemented yet, this precision hence cannot be used for QAT (only backward is broken).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels