Enforce information that can verify the validity of the bug. #1
Description
Hi, all. I am Insu Yun, an assistant professor at KAIST. First of all, I would like to thank you for your great work. I have also been experiencing similar issues and was planning to conduct similar research. Your research is more thorough than I had anticipated and is truly excellent.
I would like to add some instructions to the guideline I am considering. I would appreciate it if you can share your opinions about this.
2. Identify suitable targets for the evaluation
+ - Do not use targets that are not being actively developed
I still see many JavaScript studies using ChakraCore as their target. Unfortunately, ChakraCore has been barely maintained, with only about 20 commits applied since 2022. I believe we need to work on targets that are actively maintaining.
+ 3.1.1.5. specify whether the bug has been reported, confirmed, is a duplicate (exists in the latest version but has already been reported by other people), or has been patched.
+ 3.1.1.6 provide a tracker for the bug to validate it (e.g., a bug report ID or commit hash if it has been patched).
As you mentioned in the paper, we need to be be possible to evaluate the bugs. Thus, when reporting a new bug in the paper, we need sufficient information to track the bug. Therefore, I believe the above things should be enforced.
Thank you in advance.