Skip to content

--profile doesn't work with Python 3.12+ #258

@mistydemeo

Description

@mistydemeo

Python 3.12 introduced a new limitation preventing multiple nested profilers from running at the same time: python/cpython#110770

We're running into this limitation with warcprox. Attempting to run with --profile enabled will cause it to immediately error out with an Another profiling tool is already active exception the first time it tries to spin up a BasePostfetchProcessor thread.

We instantiate cProfiler profilers in one place - inside BasePostfetchProcessor.run:

self.profiler = cProfile.Profile()
self.profiler.enable()
self._run()
self.profiler.disable()

We do, however, additionally interact with those profilers via PooledMitmProxy, where we grab the profiler from the thread, enable it, process the request, and disable it again:

profiler = self.profilers[threading.current_thread().ident]
profiler.enable()
self._process_request_thread(request, client_address)
profiler.disable()

This could either be repeated invocations to enable() the same profiler when it's already enabled (calling .enable() twice on the same profiler raises an exception), or a call to enable() a second profiler from a different thread while a profiler from the first thread is still active.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions