It is a bit inconvenient to find and write the unique (because of hashes) paths to files. So, maybe it would be useful at least to have a copy of the result files (not all intermediate files, just the target ones). Another option is to have a file somewhere in the root of the dataset, where all existing hashes related to the dataset are stored.
p.s. these ideas are not mutually exclusive
It is a bit inconvenient to find and write the unique (because of hashes) paths to files. So, maybe it would be useful at least to have a copy of the result files (not all intermediate files, just the target ones). Another option is to have a file somewhere in the root of the dataset, where all existing hashes related to the dataset are stored.
p.s. these ideas are not mutually exclusive