Conversation
| if self.backend == "ray": | ||
| try: | ||
| from ray.util.joblib import register_ray | ||
| register_ray() | ||
| except ImportError: | ||
| NotInstalled("ray", "dev") | ||
|
|
There was a problem hiding this comment.
Is there a reason why we're not able to use ray.init() here?
There was a problem hiding this comment.
Yah we could do that. It would just require passing kwargs along to ray.init() through Runner.run then to Runner._run I thinks it's cleaner to let the user do ray.init() from their script / code. But, that brings ups a question though. Is the try except block even necessary if the user performs ray.init() in their code because they'll receive an import error well beforehand. The argument I then see for putting ray.init() here is to assist users with a customized error message that will give them a little bit more help.
There was a problem hiding this comment.
My initial guess would be that the average user will have access to a laptop and not a compute cluster. Given that the main use-case will be merely a laptop it occurs to me as more plausible that the user doesn't want to be bothered with setting up a ray-cluster locally. Especially when using this package.
We could support both, something like ray and ray-cluster as backend strings but I'm wondering if that over-complicates thing. If we had to assume one I'd favor the ray.init() route. Mainly because I imagine that's how many folks will want to use it.
@bradday4 what are your thoughts?
koaning
left a comment
There was a problem hiding this comment.
Mainly have a question on weather or not ray needs to be running as a service beforehand or if we can start ray just for our own run.
Ray does not need to be running as a service. I.e called explicitly from the CLI before hand. We can use |
This PR address issue #26 .
Things to note.
I've updated the
tests.ymlto install ray in a local environment for testing. Also created a newpytest.inifile for ignoring deprecation warnings that ray throws. This is an issue that the maintainers of joblib will deal with so I don't see any reason to clutter pytest responses with a 100+ deprecation warnings.Also made some minor modifications to the
Runnerdocstring to reflect support for ray.I've performed the unit tests locally any everything checks out.
Code coverage has one caveat being the try - except block in
Runner._runwhich catches the import error for ray. If we really wanted to run this bit of code we would need to altertests.ymlto run a specific test without ray installed, then install ray and run the normal test suite. I figured this was too much work for 5 lines of coverage.other odds and ends.
Let me know if you need me to do anything else.