Several key issues on map change labels

[qiaozhijian]

I find some inconsistency (from my perspective) of map change labels. This may mean that the labeling of map changes is a very custom work, related to each person's judgment criteria and map elements of interest. Is it necessary to further improve the labels of map change?

There are two examples: from left to right are the front view, the rendered map, and the semantic segmentation.

First, log 01bb304d7bd835f8bbef7086b688e35e__Summer_2019 at timestamp 315968879949927215. This place has been labeled as no change, but you can see that the lane lines are noticeably faded.

Second, log 1SKq8bGqPRZq6PUbFgTwVLyMZEwtiD2S__Summer_2020 at timestamp 315966949549927209. Maybe because the annotator didn't pay attention to the stop line and didn't draw it into the map, but the stop line was segmented so that the map was considered to have changed.

[qiaozhijian]

There is another example of BEV in 2S1xkgWyVhoa5zwPLz9tl2MAG0lMxj6i on 315980583459840000. Obviously, the sidewalk has disappeared, but there are still obvious sidewalk markings on the map, and this case is labeled as "no change".

Filename: 2S1xkgWyVhoa5zwPLz9tl2MAG0lMxj6i__Autumn_2020_315980583459840000_rgb_interpTruelinear_projmethodray_tracing.jpg

Filename: 2S1xkgWyVhoa5zwPLz9tl2MAG0lMxj6i__Autumn_2020_315980583459840000_vectormap.jpg

[johnwlambert]

Hi @qiaozhijian, thanks for your note. There are expected to be a few minor errors from human labeling, as more than 10,000 frames are labeled and evaluated. However, let me take a closer look at these. In the last comment, did you mean "crosswalk" instead of "sidewalk"?

[qiaozhijian]

Hi @qiaozhijian, thanks for your note. There are expected to be a few minor errors from human labeling, as more than 10,000 frames are labeled and evaluated. However, let me take a closer look at these. In the last comment, did you mean "crosswalk" instead of "sidewalk"?

Thank you for your reply. I'm sorry I made a mistake. It's "crosswalk".

Looking forward to this dataset being better. Your work has really contributed to the research community, thank you very much!