Hi Team,
We are dealing with millions of rows and current implementation of uploading to stream fails after sometime as it process the data in smaller chunks.
Can you please allow chunksize to be defaulted to your way of implementation and also allow users to change it through environment variable.
I have done this custom implementation and making it work - however it unnecessary creates a dependency of us creating a wheel file with changed code.
Let me know if you have any doubts...