HashStore-java 1.1.0 Release

[doulikecookiedough]

Summary of Changes:

• New HashStoreClient.java class which can be used to interact directly with a HashStore (and respective test class).
• New utility class in filehashstore - FileHashStoreUtility, which contains generic methods and methods shared between FileHashStore and other classes.
• Expanded functionality to call storeObject without a pid
• New Public API methods (findObject, tagObject, verifyObject) and updates to HashStore interface for overload methods.
• Reference system under the hood to manage and keep track of reference files to store, retrieve and delete objects.
• Refactored existing deleteObject(String pid) to delete an object, its reference files and all associated metadata documents. Plus new overload method deleteObject(String idType, String id) to enable deletion of object, references files and metadata documents - or just the object itself
• Refactored existing deleteMetadata(String pid) to delete all associated metadata documents for the given pid
• Refactored storeMetadata to store metadata for a given pid and formatId in a directory formed by calculating the hash of the given pid, with the document name being the hash of the formatId.
• Refactored 'ObjectMetadata' class to include a pid in its constructor and updated storeObject methods with a pid
• Updated HashStore interface javadocs and README.md

Hi @taojing2002, when you have scope - can you please help perform a code review for HashStore-java?

[doulikecookiedough]

Hi @artntek, can you please also perform a code review when you free up next?

[doulikecookiedough]

Thank you for your feedback and code review @artntek! I have incorporated most if not all of your suggestions and resolved our conversations. If you have any other comments, please let me know 🙏 .

[taojing2002]

Synchronizing the deleting methods doesn't help. If we really want to delete empty directories, we should somehow lock the file system, which is hard to do. So keeping the empty directories may be the easiest and safest way.

…

On Thu, Oct 19, 2023 at 7:20 PM Matt Jones ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ On src/main/java/org/dataone/hashstore/filehashstore/FileHashStore.java <#48 (comment)> : If these are just the prefixed two digit directories, I think it is just fine to leave them. — Reply to this email directly, view it on GitHub <#48 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5QQDHTL6KQJK6NJN4LBFLYAHNW3AVCNFSM6AAAAAA45LYXP6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTMOBZGA2TINRWG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[doulikecookiedough]

Thank you for clarifying and the feedback @taojing2002 (and @mbjones)! I believe I've addressed all your feedback, and have pushed my latest commits to the develop branch. Please let me know if you have any concerns with these changes (or any other feedback).

[doulikecookiedough]

The necessary changes required to be integrated into the Metacat client have been made. However, after further discussions, we have determined that there is still one missing piece remaining to complete the 1.0.0 release described here: Refactor deleteObject for new requirements.

Additionally, the recent changes have been tested with test data found in knbvm (test.arcticdata.io) with up to 100,000 objects which includes numerous persistent identifiers that reference the same object (cid). Testing is ongoing, with the next steps to test the entire var/metacat/data directory (400,000+ objects).