-
Notifications
You must be signed in to change notification settings - Fork 4
Description
It may happen that a large number of RESTORE threads is triggered at once, just consider requesting a restore of an investigation, that will translate into starting a RESTORE thread for each dataset in that investigation. So the number of simultaneous RESTORE threads may easily raise to a few hundreds. RESTORE operations are relative expensive in terms of I/O as they need to write many files. It has been observed that such an event may seriously degrade server performance. Furthermore, RESTORE operations are usually less urgent than WRITE (which makes sure data is saved to archive) or ARCHIVE (which may need to be done quickly to prevent hitting the size limit of main storage). So it makes sense to prioritize the latter.
Therefore I suggest there should be a configuration option that would set a limit to the number of RESTORE threads that are started simultaneously. The exceeding RESTORE requests would remain waiting in the queue and would be carried out as soon as there is free capacity, so this shouldn't hurt much.