An alternative approach to temporarily storing all model URLs on the shared filesystem would be to have an endpoint on the loadbalancer that can accept a model URL (e.g. via HTTP POST) inside of each hyperqueue job. This would allow for the storage of URLs in memory rather than requiring an expensive file I/O operation.
An alternative approach to temporarily storing all model URLs on the shared filesystem would be to have an endpoint on the loadbalancer that can accept a model URL (e.g. via HTTP POST) inside of each hyperqueue job. This would allow for the storage of URLs in memory rather than requiring an expensive file I/O operation.