Question about label free setting

[Pixie8888]

Hi,

What is the label set used in computing eqn. 1. Is it the whole set of classes in ScanNet when trained on ScanNet?

In addition, since eqn.1 uses label, I don't think it is the strictly "label free" setting.

[ldkong1205]

Hi @Pixie8888, thanks for asking this question!

Regarding Q1:
Yes, when trained on ScanNet, we use the class names from ScanNet as the input text prompts to obtain CLIP’s text embeddings.

Regarding Q2:
We harness the image encoder from CLIP to derive dense pseudo-labels. It's essential to highlight that while these pseudo-labels are sourced directly from CLIP, our framework under the "label-free" setting doesn't require any additional 2D or 3D segmentation annotations.