SmanticKITTI - Baseline results are not reproducible

[chreisinger]

Hello and thanks for the great work!

I have now tried the framework and trained the baseline (random initialization) with 0.1% and 1% of the data on SemanticKITTI with the MinkUNET. However, I get much lower values. For 0.1% of the data I get 20 mIOU (instead of the reported 30%) and for 1% of the data I get 35.7% mIOU (instead of the reported 47%). Do you have any idea whats the issue here?

Best regards,
Christian

[aboulch]

Hello,
Thanks for reporting this.
In the meantime, can you check that you are really training on SemanticKITTI, these scores correspond to those I obtained on nuScenes.
I may have done a mistake in the config files, I will check the code.

Best regards,