Skip to content

Commit 6d169f6

Browse files
authored
[build] Do not do process isolation on win or mac (#3441)
multiprocess defaults to spawn on macs and fork on linux, and I think sphinx makes everything in conf.py be in the main module even if you import from another file so its not picklable, so you get errors like ``` Handler <function generate_gallery_rst at 0x31029b920> for event 'builder-inited' threw an exception (exception: Can't pickle <function call_fn at 0x107e14860>: attribute lookup call_fn on __main__ failed) ``` or if you try to put call_fn in a different file: ``` Handler <function generate_gallery_rst at 0x17c59b920> for event 'builder-inited' threw an exception (exception: Can't pickle <function generate_file_rst at 0x1412322a0>: it's not the same object as sphinx_gallery.gen_rst.generate_file_rst) ``` Mac does have fork, but MPS doesn't, and windows doesn't support fork, so don't monkey patch if on either of these platforms Testing: Ran `make html-noplot` and saw that it got past the place it originally errored. Not testable in CI because we don't currently have mac builds in CI Notes: multiprocess + mac wants spawn b/c MPS stuff -> needs to be picklable -> some inputs to generate file rst not pickleable Could probably manually pickle with cloudpickle since sphinx gallery later versions use joblib which use cloudpickle but this seems like overkill
1 parent 876c563 commit 6d169f6

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

conf.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,10 @@ def wrapper(*args, **kwargs):
8585
raise RuntimeError(f"Error in subprocess: {result}")
8686
return wrapper
8787

88-
sphinx_gallery.gen_rst.generate_file_rst = call_in_subprocess(sphinx_gallery.gen_rst.generate_file_rst)
88+
# Windows does not support multiprocessing with fork and mac has issues with
89+
# fork so we do not monkey patch sphinx gallery to run in subprocesses.
90+
if os.getenv("TUTORIALS_ISOLATE_BUILD", "1") == "1" and not sys.platform.startswith("win") and not sys.platform == "darwin":
91+
sphinx_gallery.gen_rst.generate_file_rst = call_in_subprocess(sphinx_gallery.gen_rst.generate_file_rst)
8992

9093
try:
9194
import torchvision

0 commit comments

Comments
 (0)