Limit zip export resources size and add option to exclude attachments

[tylerjmchugh]

Currently there is no limit on the size of zip exports.

Consider a catalogue of 1000 records each with a 1GB attachment/resource. If a user selects all the records and attempts to export as zip, the system will try to create a 1000GB zip file which will cause performance issues.

This PR aims to solve this issue by implementing a size limit.

An option was added to the system settings:

Additionally the ui was updated to disable the zip export and a new option to export as zip without the attachments is added:

Checklist

• I have read the contribution guidelines
• Pull request provided for main branch, backports managed with label
• Good housekeeping of code, cleaning up comments, tests, and documentation
• Clean commit history broken into understandable chucks, avoiding big commits with hundreds of files, cautious of reformatting and whitespace changes
• Clean commit messages, longer verbose messages are encouraged
• API Changes are identified in commit messages
• Testing provided for features or enhancements using automatic tests
• User documentation provided for new features or enhancements in manual
• Build documentation provided for development instructions in README.md files
• Library management using pom.xml dependency management. Update build documentation with intended library use and library tutorials or documentation

[ianwallen]

Should zip export and export as zip without the attachments generate different filenames?
i.e. should zip export without attachments generate a file called *-noattachments.zip ?
What are others thoughts on this?

[juanluisrp]

We need a way of knowing if the zip doesn't have attachments because it would be too big or the attachment export is disabled, or because the metadata actually don't have any attachments. Using a different name could be a fix for that.

[tylerjmchugh]

Should zip export and export as zip without the attachments generate different filenames?
i.e. should zip export without attachments generate a file called *-noattachments.zip ?

We need a way of knowing if the zip doesn't have attachments because it would be too big or the attachment export is disabled, or because the metadata actually don't have any attachments.

@ianwallen @juanluisrp

Maybe something like *-metadata-only.zip or *-attachments-excluded.zip to better indicate the reason there are no attachments?

There are really only two cases where there are no attachments

• Normal export but metadata has no attachments
  • Would produce: 0e1943d6-64e8-4430-827c-b465c3e9e55c.zip
• Export without attachments
  • Forced from the UI if the size limit is exceeded
  • Would produce: 0e1943d6-64e8-4430-827c-b465c3e9e55c-attachments-excluded.zip

[juanluisrp]

I haven't checked all the code in PR yet, but I see that AttachmentsExportLimitExceededException is managed in the GlobalExceptionHandler, returning an error message. However the description says that if the size of the attachments is larger than the allowed size the ZIP will be generated without attachments. Is it actually working that way?

[tylerjmchugh]

@juanluisrp Sorry for the confusion.

The description was updated but the screenshot above and the documentation was missed.

No the attachments will not be excluded if they exceed the limit. An exception is thrown instead.

Here is the updated screenshot:

[ianwallen]

Maybe something like *-metadata-only.zip or *-attachments-excluded.zip to better indicate the reason there are no attachments?

Maybe follow the same naming as the selection option

*-without-attachments.zip