About INT4 Convolution

[Akimoto-Cris]

Hi @wangmaolin,

I find your work very interesting.

I saw your implementation of int4_conv in cutlass extension, and found that the weight is unsigned:

niti/pytorch/cutlass-extension/cutlassconv_kernel.cu

Line 857 in d097335

cutlass::TensorRef<ElementInputB, LayoutInputB> weight_ref(reinterpret_cast<ElementInputB *>(weight.data<uint8_t>()), LayoutInputB::packed(filter_size));

.

Did you test this function in real CNNs? Since weights are normally zero-centered, any conversion on weights should be done before using this int4_conv function in order to get the right results?

Thanks in advance.