Skip to content

Reproducing Results from Table 2 #40

@wuwuwuxxx

Description

@wuwuwuxxx

Thank you for sharing your insightful paper and making the code publicly available. I've been working on reproducing the results from Table 2, but I'm encountering some discrepancies compared to the reported values.

Here are the results I obtained using the provided configurations:

{"rois.thing.macc1": 0.1817628860473633, "rois.thing.macc5": 0.33184242248535156, "rois.stuff.macc1": 0.13718507874686764, "rois.stuff.macc5": 0.36789228331367924, "crops.thing.macc1": 0.6843925476074219, "crops.thing.macc5": 0.8644012451171875, "crops.stuff.macc1": 0.22904190927181603, "crops.stuff.macc5": 0.5747173957104953, "maskpool.thing.macc1": 0.17424376010894777, "maskpool.thing.macc5": 0.32028732299804685, "maskpool.stuff.macc1": 0.16302821321307487, "maskpool.stuff.macc5": 0.40581411685583724}

I've carefully followed the instructions in the repository and used the same configurations, but my results consistently differ from those in the paper. Could you please advise on:

Any potential implementation details that might affect reproducibility?
Whether there are additional hyperparameters or environment settings that weren't explicitly mentioned?

I'd be grateful for any insights that could help resolve this difference. Thank you for your time and consideration!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions