Error deleting backup despite using version 3.16.2 of Barman ( Content-MD5 is missing).

[arkadyj]

Hi
I'm still having trouble deleting backup files despite using the latest version of Barman 3.16.2.
Boto3 version: 1.41.2
Botocore version: 1.41.2

After returning Boto3 to 1.35.99, everything works fine.

postgres@test-trixie-01d-1:/$ barman-cloud-backup-list --cloud-provider aws-s3 --endpoint-url https://osf01189.pzu.pl:9021  s3://s3backup//AREKTEST/DEV test-trixie-01d -v
2025-11-26 12:04:22,922 [1148] INFO: Found credentials in environment variables.
Backup ID           End Time                 Begin Wal                     Archival Status  Name
20251126T113619     2025-11-26 11:36:22      000000010000000000000003                       backup-20251126113618
20251126T120357     2025-11-26 12:04:01      000000010000000000000008                       backup-20251126120356

postgres@test-trixie-01d-1:/$ barman-cloud-backup-delete --cloud-provider aws-s3 --endpoint-url https://osf01189.pzu.pl:9021 -b "20251126T120357" s3://s3backup//AREKTEST/DEV test-trixie-01d -v

2025-11-26 12:04:39,222 [1151] INFO: Found credentials in environment variables.
2025-11-26 12:04:39,552 [1151] INFO: Found file from backup '20251126T120357' of server 'test-trixie-01d': AREKTEST/DEV/test-trixie-01d/base/20251126T120357/data.tar
2025-11-26 12:04:39,584 [1151] ERROR: Could not delete backup 20251126T120357: An error occurred (InvalidRequest) when calling the DeleteObjects operation: Content-MD5 is missing.

postgres@test-trixie-01d-1:/$ barman-cloud-backup-delete --version
barman-cloud-backup-delete 3.16.2

postgres@test-trixie-01d-1:/$ pip show boto3
WARNING: The directory '/var/lib/postgresql/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Name: boto3
Version: 1.41.2
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache-2.0
Location: /usr/local/lib/python3.13/dist-packages
Requires: botocore, jmespath, s3transfer
Required-by:

postgres@test-trixie-01d-1:/$ pip show botocore
WARNING: The directory '/var/lib/postgresql/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Name: botocore
Version: 1.41.2
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache-2.0
Location: /usr/local/lib/python3.13/dist-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
postgres@test-trixie-01d-1:/$

[martinmarques]

Which object store are you using?

It seems like each vendor has decided to write their own "missing MD5" error message. See this: #1124 (comment)

[arkadyj]

We use DELL EMC Isilon with an S3 resource provisioned. We're in the process of migrating our backups to Minio, where we don't have these issues.

[martinmarques]

@alexeyshishkin-edb you have more knowledge about these DELL storage. Was there a software version that was required for this to work?

[alexeyshishkin-edb]

Minimum Supported Code Version of ECS Software is 3.8.1 according to Dell Support

[martinmarques]

We have an internal task to modify the way we check for this "faulty software" object storage, which will likely fix this issue. We don't know if this will be in 3.17 or in a patch release for 3.16, but we are planning for the next release.

[martinmarques]

This has been fixed in 3.16.2

[albundy83]

Are you sure it is in 3.16.2 ?

[barthisrael]

This issue has been closed by mistake.

3.16.2 did include a patch for improving check coverage for non-compliant providers, but that was not enough.

3.17.0 will include a patch with a broader check for the Content-MD5 issues, in an attempt to cover more non-complaint providers.

[albundy83]

Ah thanks a lot for clarification :)

[barthisrael]

Fixed in 3.17.0