Skip to content
This repository was archived by the owner on Oct 5, 2025. It is now read-only.

S3 credentials enhancements and fix URL parsing#76

Merged
groldan merged 1 commit intomainfrom
s3/optional_default_credentials_provider
Sep 8, 2025
Merged

S3 credentials enhancements and fix URL parsing#76
groldan merged 1 commit intomainfrom
s3/optional_default_credentials_provider

Conversation

@groldan
Copy link
Contributor

@groldan groldan commented Sep 8, 2025

S3 credential enhancements:

  • Add USE_DEFAULT_CREDENTIALS_PROVIDER parameter (default: false for public access)
  • Add DEFAULT_CREDENTIALS_PROFILE parameter for custom AWS profiles
  • Use AnonymousCredentialsProvider for unsigned requests to public S3 resources
  • Enable reuseLastProviderEnabled and asyncCredentialUpdateEnabled for server optimization
  • Add comprehensive Javadoc for all credential-related builder methods

Restrict S3RangeReaderProvider to files only and add public S3 access support

  • S3RangeReaderProvider now requires both non-empty bucket AND key to claim URLs
  • This ensures RangeReader only handles URLs pointing to specific files, not bucket roots
  • Custom domain URLs like demo-bucket.protomaps.com now fall back to HttpRangeReader
  • Updated tests to reflect that bucket root URLs should use HTTP, not S3 client
  • Maintains support for legitimate S3-compatible services (MinIO, etc.) with file paths

S3 credential enhancements:

- Add USE_DEFAULT_CREDENTIALS_PROVIDER parameter (default: false for public access)
- Add DEFAULT_CREDENTIALS_PROFILE parameter for custom AWS profiles
- Use AnonymousCredentialsProvider for unsigned requests to public S3 resources
- Enable reuseLastProviderEnabled and asyncCredentialUpdateEnabled for server optimization
- Add comprehensive Javadoc for all credential-related builder methods

Restrict S3RangeReaderProvider to files only and add public S3 access support

- S3RangeReaderProvider now requires both non-empty bucket AND key to claim URLs
- This ensures RangeReader only handles URLs pointing to specific files, not bucket roots
- Custom domain URLs like demo-bucket.protomaps.com now fall back to HttpRangeReader
- Updated tests to reflect that bucket root URLs should use HTTP, not S3 client
- Maintains support for legitimate S3-compatible services (MinIO, etc.) with file paths
@groldan groldan merged commit d1a0d79 into main Sep 8, 2025
20 checks passed
@groldan groldan deleted the s3/optional_default_credentials_provider branch September 8, 2025 17:24
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant