-
Notifications
You must be signed in to change notification settings - Fork 8
Description
Well, POAP just hit the fan.
I'm attempting to optimize a function which involves running a black box model 3 times and then checking the combined fit of the results. The model is launched from win32com (COM API), and the call waits until the model finishes. I need to run three of these in parallel, so I'm instantiating three win32com.clients, split using multiprocessing.Process.
All is well and good with this objective function. I am using the SerialController from POAP and it works fine if I only run 1 of the 3 models in the objective function, and bypass the multiprocessing.
However, when turning multiprocessing back on, that's when POAP hits the fan...
# toplevel.py (runs strategy)
The following print statements appear in this order:
- correct prints from start of objective func
- somehow imports a library 3 times which is only imported in toplevel.py (this lib prints when it imports)
- prints statements from toplevel.py 3 times
- prints the parameters used from within the objective func 3 times
- POAP silently prints a file access error that occurs when the objective func is trying to edit the same file 3 times
- prints the correct setup statements from FIRST run only
- freeze support err:
multiprocessing\spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
- print statements from one objective func that should happen AFTER running the model
- repeats forever, regardless of specified max evaluations
the model never actually runs
It seems that serialController is still using threading, which is going bananas when it reaches a multiprocessing statement in the objective function, including somehow rerunning the very same toplevel.py file that I'm running. How can I disable this so that nothing executes in parallel except for my model calls within the objective function?