Rate-limit requests made by couchbackup #92
Description
CouchBackup, by default, goes as fast as it can within the contraints of the parallelism setting (which controls the concurrent connections to a server).
Instead, it'd be nice for customers to be able to control the number of requests made per second. At least, the number of requests initiated per second, which is simpler to implement!
The approach I'd take to this would be a token bucket approach. As each item is pulled off the queue, block it until tokens are available. As async is already a dep, using its forever or retry function probably come in useful for blocking.
The complexity here is to maintain first-in-first-out, to avoid resulting in requests made early in the backup process being starved for long periods. I'd definitely investigate methods to retain request ordering, otherwise pretty odd behaviour may happen (or, at least, hard to understand!). The queue has an unshift method which could be used to place tasks back on the queue, which combined with a setTimeout to give us a delay before unshifting might work. I'd avoid the queue's pause and resume.
I'd strongly suggest a refill strategy based on calling a refill function for every allow (similar to golang's rate) rather than a setTimeout based approach (like here) as it'll be more predictable.