Skip to content

Conversation

@KefanXIAO
Copy link

In order to use the S3, add a S3 filesystem similar to the GCS one. The S3 client and the APIs like GetObject are also applied similar to the GCS one. Tested with my S3 bucket and it works with both integrity and good efficiency.

@google-cla
Copy link

google-cla bot commented Dec 19, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@KefanXIAO
Copy link
Author

Just signed the CLA. PTAL

@KefanXIAO
Copy link
Author

Hi @charlesbeattie and @Augustin-Zidek , would you mind taking a look at this PR? I'm adding a functionality for S3.

@lucasb-eyer
Copy link

I recommend that you use absl::call_once instead of the static to make it thread-safe:

--- a/src/file/file_systems/s3/s3_file_system.cc
+++ b/src/file/file_systems/s3/s3_file_system.cc
@@ -22,6 +22,7 @@
 #include <vector>
 
 #include "absl/algorithm/container.h"
+#include "absl/base/call_once.h"
 #include "absl/base/no_destructor.h"
 #include "absl/base/nullability.h"
 #include "absl/functional/function_ref.h"
@@ -217,12 +218,11 @@ std::shared_ptr<Aws::S3::S3Client> S3FileSystem::Client() const {
   ClientFactory client_factory = client_factory_;
   if (client_factory == nullptr) {
     client_factory = [] {
-      static Aws::SDKOptions options;
-      static bool initialized = false;
-      if (!initialized) {
+      static absl::once_flag aws_init_once;
+      absl::call_once(aws_init_once, []() {
+        static Aws::SDKOptions options;
         Aws::InitAPI(options);
-        initialized = true;
-      }
+      });
       return std::make_shared<Aws::S3::S3Client>();
     };
   }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants