Single PUT when uploading single buffer-files to S3 (version 2)

[carlopi]

This is the same logic of #106, but to me more clear and a crafted test that should show the change in behaviour.

@samansmink, thanks for the feedback, but you'll have to review also this one, sorry for the noise.

---

We currently follow always same path on uploading to S3-like storage:

• 1 POST to start upload
• 1 or more PUT requests to upload buffers
• 1 POST to finalize upload

While this is general and battle tested, there is an improvement, we could check the count of buffers to be uploaded, and if that is at 1 we could perform a single PUT (moving from 3 requests over the network to a single one).

This is particularly significant for writes to data lakes in general and Iceberg in particular, due to the fact that both JSON and AVRO files have to be uploaded (and that means a INSERT INTO <table> VALUES () costs currently 10+ sequential network requests)

Next up: figuring out how to cut the HEAD request currently performed when opening a FileHandle, that could properly move to single request.

For the reviewer(s): I am assuming we always fully upload a file, and never just a single part of an existing file. It would be much better to check that explicitly, unsure if there is else needed.

[Tmonster]

LGTM! Thanks

[carlopi]

I would also for merging this AND bumping httpfs in duckdb/duckdb to have also this, but we need to complete a conversation with Sam.

Basically: should this be available by default in 1.4.1 or when? Depending on the answer, then there are different strategies to roll this out.