Run evaluation only?

[taylorwwebb]

Is there a way to run the model in evaluation mode only using the meta-trained model provided in the 'Model Zoo' section? The instructions in the readme only describe how to train the model from scratch. I am specifically interested in performing the evaluation for the VOC dataset.

[GuangxingHan]

Please refer to this issue for the evaluation script #2.
Thanks!

[taylorwwebb]

Thank you very much for your response. I am having some difficulty determining which data were used for the query/test images during meta-learning and meta-testing on the VOC dataset (i.e., the results reported in the top part of table 5 in the paper). I have performed the data preparation steps as instructed in the README file. I see files in ./dataset/pascal_voc/ that seem to contain the necessary information about the few-shot training examples for meta-learning (e.g., ./dataset/pascal_voc/voc_2007_trainval_all1_10shot.pkl), but I am unable to determine which data were used as query images. The config file for the third and final step of meta-training seems to refer to a test dataset named 'voc_2007_test_all1', but I am not able to find any files with this name. Is there any way to determine which data were used as query images for meta-testing?

[GuangxingHan]

Thank you for your interest in our work.

You can find the dataset 'voc_2007_test_all1' here (https://github.com/GuangxingHan/Meta-Faster-R-CNN/blob/main/meta_faster_rcnn/data/datasets/builtin.py#L79)

[taylorwwebb]

Thank you very much for your response. After preparing the data following the instructions, there seem to be files that contain the necessary information about the few-shot training examples. For example, the file voc_2007_trainval_all1_5shot.pkl contains information for 5 examples for each class. But I do not see any file with a name that contains the text 'voc_2007_test_all1' in the ./datasets/pascal_voc/ directory, and I cannot determine which data were used as query images (either during meta-training or meta-testing). Is there any way to determine the content of the datasets used for meta-training and meta-testing (i.e. to access these datasets without actually running the entire metalearning pipeline)?