DataPackageManager: Writing data greater than 78GB to an S3 bucket using mount-s3 fails

[servilla]

Writing data greater than 78GB to an S3 bucket mounted to /home/pasta/local/s3 using mount-s3 fails. This has been confirmed with both Java and Python attempts to download (streaming/chunking) and write the data.

Java:

java.io.IOException: File too large
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
at edu.lternet.pasta.dml.download.DownloadHandler.writeRemoteInputStreamIntoDataStorage(DownloadHandler.java:1073)
at edu.lternet.pasta.dml.download.DownloadHandler.getContentFromSource(DownloadHandler.java:575)
at edu.lternet.pasta.dml.download.DownloadHandler.run(DownloadHandler.java:350)
at java.lang.Thread.run(Thread.java:750)

Python:

Progress: 78 GB downloaded...
Traceback (most recent call last):
  File "/home/pasta/python/big/download_test.py", line 30, in download_large_tif
    f.write(chunk)
    ~~~~~~~^^^^^^^
OSError: [Errno 27] File too large

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pasta/python/big/download_test.py", line 44, in <module>
    download_large_tif()
    ~~~~~~~~~~~~~~~~~~^^
  File "/home/pasta/python/big/download_test.py", line 26, in download_large_tif
    with open(filename, 'wb') as f:
         ~~~~^^^^^^^^^^^^^^^^
OSError: [Errno 27] File too large