When you are sure that there are many files that are duplicates and only a few that are unique, and there are many big files,
these steps are often useless. Probably it is faster to go straight to the full file check.
For example:
rdfind -checksum md5 -makehardlinks true Mese.00 Mese.01
Now scanning "Mese.00", found 754116 files.
Now scanning "Mese.01", found 751807 files.
Now have 1505923 files in total.
Removed 133484 files due to nonunique device and inode.
Total size is 2431878969608 bytes or 2 TiB
Removed 5960 files due to unique sizes from list. 1366479 files left.
Now eliminating candidates based on first bytes: removed 8578 files from list. 1357901 files left.
Now eliminating candidates based on last bytes: removed 1727 files from list. 1356174 files left.
Now eliminating candidates based on md5 checksum: ...
When you are sure that there are many files that are duplicates and only a few that are unique, and there are many big files,
these steps are often useless. Probably it is faster to go straight to the full file check.
For example:
rdfind -checksum md5 -makehardlinks true Mese.00 Mese.01
Now scanning "Mese.00", found 754116 files.
Now scanning "Mese.01", found 751807 files.
Now have 1505923 files in total.
Removed 133484 files due to nonunique device and inode.
Total size is 2431878969608 bytes or 2 TiB
Removed 5960 files due to unique sizes from list. 1366479 files left.
Now eliminating candidates based on first bytes: removed 8578 files from list. 1357901 files left.
Now eliminating candidates based on last bytes: removed 1727 files from list. 1356174 files left.
Now eliminating candidates based on md5 checksum: ...