About the termnology of Replicability & Reproducibility

[VVingerfly]

HI, Thanks for the awesome work. It's really helpful and useful for CG researchers.

I am not an English speaker. And I am confused about the termnology of Replicability & Reproducibility.

I searched on the Internet, and found a definition from The American Statistical Association (ASA)

Reproducibility: A study is reproducible if you can take the original data and the
computer code used to analyze the data and reproduce all of the numerical findings
from the study. This may initially sound like a trivial task but experience has shown that
it’s not always easy to achieve this seemingly minimal standard.

Replicability: This is the act of repeating an entire study, independently of the original
investigator without the use of original data (but generally using the same methods).

But from the abstract of the paper

This process is called “replicability” when using the original authors’ artifacts (e.g., code), or “reproducibility” otherwise (e.g., re-implementing algorithms).

It seems that the two definitions are opposite? Could you help me to solve my confusion?
Thanks a lot!

B.R.

[dcoeurjo]

Thanks @VVingerfly for the feedback.

Actually, setting the proper terminology was not easy (mostly because of the confusion between replicability and reproducibility you mention). We definitely agree that opposite usages of these terms may be found in other communities.

In the paper (and the website), we have followed the ACM definition (https://www.acm.org/publications/policies/artifact-review-badging) which was also used for the http://www.replicabilitystamp.org initiative for instance (replicability=using authors' code)

[VVingerfly]

Thanks @dcoeurjo a lot for the quick reply! And thanks for providing the source of the definitions, which is very useflul for understanding these terminologies.

[nmellado]

Let’s keep this issue open, as it might be a good read for other users.

[dcoeurjo]

Let's keep it open to remind us to clarify the def (with the ACM link) in the project description (and close it when it's done, what do you think @nmellado .

[nmellado]

Yes, that was my second thought.
We will add here the link pointing to the page/paragraph commenting this aspect in the project description.