Some questions about the results.

[ybai62868]

Hi,
Thanks for the good work!
Recently, I run the code in your repo (kernel benchmark) and find the following results.

I am curious about the meaning of "TotalError".
It seems that other than cublas_*, the rest of the implementaiton is not correct?

[Summer-Summer]

Thanks for your question!
The total error here means the sum of the absolute errors for all the elements within the output matrix compared to cuBLAS, which is calculated by the following function:
double ComputeTotalError(half* CuBlas, half* Other, int m, int n) { double totalError = 0.0; for (int i = 0; i < m * n; i++) totalError += fabs(__half2float(CuBlas[i]) - __half2float(Other[i])); return totalError; }
Thus, the error for cuBLAS would always be 0. It does not mean that other implementations are producing wrong results, but there are some tiny differences, e.g. may caused by the change of the order for inner dot accumulation. For example, the overall error is 408.53 for an output matrix in the shape of 7168 * 8 in your case. It means on average each output value (a single FP16 value) differs from cuBLAS's output by 408.3/(7168 * 8) = 0.007.

[fan1997]

Hi, thanks for the question and quick answer, and why there is a warning about "[Warning]: The matrix has been pruned in a wrong way. cusparseLtMatmul will not provide correct results" ?

[Summer-Summer]

Hi, thanks for the question and quick answer, and why there is a warning about "[Warning]: The matrix has been pruned in a wrong way. cusparseLtMatmul will not provide correct results" ?

cusparseLtMatmul is used in SparTA where the 2:4 sparse Tensor cores are used. This warning occurs occasionally for some MatMul shapes when a sparse matrix is transformed into a 2:4 format. However, I did not see any wrong outputs nor did this warning affect the MatMul performance. Thus, I choose to ignore this warning and believe it is caused by some bugs within the cusparseLT library. If you find any method to solve this warning, please let me know.