cache directory persistence in docker containers

[RZachLamberty]

in somewhat limited experience, I've found that one of the most important features of the dl libraries has been the caching / model checkpointing components. for example, shipping a trained model requires, at some point, persisting at least the weights, if not a pickled repr. of the entire model.

in practice I've always done this by creating a system-wide cache location a la /cache, or /var/cache/eri, though obviously the exact path isn't that important.

I would argue this is not stuff we should be putting in the /data directory simple on the basis of separation of concerns -- what we are putting here are artifacts and products rather than inputs. for space constraint reasons

I propose we basically repeat the /data setup with a new directory for caching results

[RZachLamberty]

a similar and perhaps combinable use case: ozan's notebooks