object-ID alignment mismatch between the public caption JSON files and the instance IDs produced by the official ScanNet++ toolbox preprocessing

[zhaoyibo0422-code]

Hi, thanks for open-sourcing ExCap3D.

I am trying to reproduce the ScanNet++ captioning pipeline using the public code and the public ExCap3D caption release. I found what looks like an object-ID alignment mismatch between the public caption JSON files and the instance IDs produced by the official ScanNet++ toolbox preprocessing.

What I did:

1. Prepared full .pth files using the official ScanNet++ toolbox:
   python -m semantic.prep.prepare_training_data semantic/configs/prepare_training_data.yml
2. Sampled points with sample_pth.py
3. Converted sampled .pth to Mask3D .npy with datasets.preprocessing.scannetpp_pth_preprocessing.py

From the ScanNet++ toolbox code (semantic/transforms/mesh.py), vtx_instance_anno_id is assigned from:

instance_anno_id_multi[inst_mask, new_label_position] = instance['objectId']
sample['vtx_instance_anno_id'] = instance_anno_id
I verified that:

full_pth_data_normals and pth_data_sampled_normals have the same instance IDs
sampled .pth and final Mask3D .npy also have the same instance IDs
However, the public ExCap3D caption JSON keys do not match these IDs.

Concrete example: scene 286b55a2bf

In segments_anno.json, relevant objectIds are:
toilet -> 51
sink -> 50
soap dispenser -> 49
door -> 63
cabinet -> 72
In the public ExCap3D caption JSON, the corresponding captions are keyed by:
13 -> toilet caption
14 -> sink caption
15 -> soap dispenser caption
32 -> door caption
33 -> cabinet caption
I also checked:

PTH vs NPY for this scene: shared=23, pth_only=0, npy_only=0
PTH vs caption JSON for this scene: shared=0, pth_only=23, caption_only=13
On the validation split, this leads to many mismatches:

caption-only inst ids: 191 / 1135
suspicious overlap inst ids: 218 / 944
My questions are:

Are the public ExCap3D caption JSON keys expected to match ScanNet++ segGroups[].objectId?
If not, what ID do the public caption JSON keys correspond to?
Is there an official mapping script or a specific ScanNet++ preprocessing version needed to align the public captions with the released code?
Are the internal scene_captions files used in the training scripts different from the public excap3d-release JSON files?
If helpful, I can attach the single-scene evidence for 286b55a2bf.

Thanks.

[cy94]

Hi, the instance IDs should be the same as in the released data. Are you using the latest v2 version of the ScanNet++ data? And is the issue only in the validation set?

[zhaoyibo0422-code]

Thanks for the quick reply.

Yes, I am using the latest public ScanNet++ v2 release and the official toolbox pipeline:

• data_root: /mnt/Datasets/scannetpp/data
• labels_path: /mnt/Datasets/scannetpp/metadata/semantic_benchmark/top100.txt
• instance_labels_path: /mnt/Datasets/scannetpp/metadata/semantic_benchmark/top100_instance.txt
• mapping_file: /mnt/Datasets/scannetpp/metadata/semantic_benchmark/map_benchmark.csv
• list_path: /mnt/Datasets/scannetpp/splits/nvs_sem_train_val.txt

I generated the full .pth with:
python -m semantic.prep.prepare_training_data semantic/configs/prepare_training_data.yml

Then I ran sample_pth.py and converted to Mask3D .npy.

From the toolbox code, vtx_instance_anno_id is assigned from segGroups[].objectId, and I verified:

• full .pth and sampled .pth have the same instance IDs
• sampled .pth and final .npy also have the same instance IDs

So the mismatch seems to be between the public ExCap3D caption JSON keys and segGroups[].objectId in the public v2 ScanNet++ data.

Concrete example: scene 286b55a2bf

• in scans/segments_anno.json:
  • toilet -> objectId=51
  • sink -> objectId=50
  • soap dispenser -> objectId=49
  • door -> objectId=63
  • cabinet -> objectId=72
• but in the public ExCap3D caption JSON for the same scene, the corresponding captions are keyed as:
  • 13 -> toilet caption
  • 14 -> sink caption
  • 15 -> soap dispenser caption
  • 32 -> door caption
  • 33 -> cabinet caption

So at least for this scene, the caption keys do not match objectId.

I first noticed it on val, but it does not seem val-only. Across the public train+val caption release, I found mixed scene-level key patterns rather than a single consistent mapping.

Could you clarify whether the public caption JSON keys are expected to match segGroups[].objectId, or whether they correspond to another scene-local ID (e.g. segGroup index / instance_ndx)? If a remapping step is required for the public release, could you share the official procedure?Duplicate of #

[zhaoyibo0422-code]

From my checks, the caption JSON keys do not behave like a single global ID field from the raw public ScanNet++ v2 annotations. Across scenes, they look mixed: some match objectId, some match segGroup index, and some match segGroup index + 1.

So I suspect “the released data” in your comment may refer to the released processed ExCap3D/Mask3D data rather than raw ScanNet++ v2 + self-generated preprocessing.

Could you clarify which exact instance-id field in the released processed data is expected to match the public caption JSON keys?

• segGroups[].objectId / vtx_instance_anno_id
• segGroup index / vtx_instance_labels
• or another released remapped scene-local instance id