Redivis unresponsive and browser freezing when importing from GCS

[savannahmccoy]

Describe the bug
When importing multiple tables simultaneously from Google Cloud Storage (GCS) into Redivis, I experienced site unresponsiveness and my browser freezing. I was unable to click on any buttons or links on the page and I was unable to scroll. I couldn't change tabs and had to force quit the browser an open a new one. A few tables froze for hours (showed no progress) during the transfer but eventually imported overnight.

The folder structure is as follows:

There is a bucket that contains 185 directories, each with 38 - 41 files within them. For each table, I would import the data with a wildcard path that indicated iterating over all the folders to get the desired file name for the table. For example:

bucket-name/*/table_name.CSV

Each file is no more than 500MB. Some files have hundreds of thousands of rows, some only have a few thousand. The slowness would occur after importing 3 or more tables simultaneously. Please let me know if you need additional information

Steps to reproduce

1. Create a new table for a dataset
2. Click the 'Data Source' drop down menu and select Google Cloud Storage
3. Enter a path with a wildcard at the second level
4. Click import data
5. Repeat this process such that multiple tables are importing at the same time
6. Observe slowness of browser, unresponsiveness of site

Browser
Both Chrome and Safari

Expected behavior
Table data imports in the background, while the front end remains uneffected

Any additional context
Stanford GSB Library Organization

[imathews]

Thank you for bringing this to our attention @savannahmccoy. We'll look into this, and try to get it sorted soon. So that you're aware, once the import job has been created, you can close your browser without issue (everything happens on our backend).

I imagine the browser freezing is due to the constant progress updates across thousands of uploads.