-
Notifications
You must be signed in to change notification settings - Fork 65
Description
Python 3.12 introduced a new limitation preventing multiple nested profilers from running at the same time: python/cpython#110770
We're running into this limitation with warcprox. Attempting to run with --profile enabled will cause it to immediately error out with an Another profiling tool is already active exception the first time it tries to spin up a BasePostfetchProcessor thread.
We instantiate cProfiler profilers in one place - inside BasePostfetchProcessor.run:
Lines 106 to 109 in 37e4cd5
| self.profiler = cProfile.Profile() | |
| self.profiler.enable() | |
| self._run() | |
| self.profiler.disable() |
We do, however, additionally interact with those profilers via PooledMitmProxy, where we grab the profiler from the thread, enable it, process the request, and disable it again:
warcprox/warcprox/mitmproxy.py
Lines 753 to 756 in f7dd1e7
| profiler = self.profilers[threading.current_thread().ident] | |
| profiler.enable() | |
| self._process_request_thread(request, client_address) | |
| profiler.disable() |
This could either be repeated invocations to enable() the same profiler when it's already enabled (calling .enable() twice on the same profiler raises an exception), or a call to enable() a second profiler from a different thread while a profiler from the first thread is still active.