Segmentation fault

[Gnoale]

Hello,
I get Fatal Python error: Segmentation fault when running from pypprof import net_http
I tried to dig into it but have not much experience with low level stuff and C.

I found using export PYTHONFAULTHANDLER=1 to print the stack trace during the segfault, but am unable to figure what's happening, I am using Python 3.8.5.

14:51 $ python3 -m unittest test/test_net_http.py 
Fatal Python error: Segmentation fault

Current thread 0x00007fdc7aa3b740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1101 in create_module
  File "<frozen importlib._bootstrap>", line 556 in module_from_spec
  File "<frozen importlib._bootstrap>", line 657 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1042 in _handle_fromlist
  File "/home/guillaumenoale/git/pypprof/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 48 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 783 in exec_module
  File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
    File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1042 in _handle_fromlist
  File "/home/guillaumenoale/git/pypprof/lib/python3.8/site-packages/zprofile/profile_pb2.py", line 21 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 783 in exec_module
  File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1042 in _handle_fromlist
  File "/home/guillaumenoale/git/pypprof/lib/python3.8/site-packages/zprofile/builder.py", line 20 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 783 in exec_module
  File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1042 in _handle_fromlist
  File "/home/guillaumenoale/git/pypprof/lib/python3.8/site-packages/zprofile/cpu_profiler.py", line 19 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 783 in exec_module
  File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "/home/guillaumenoale/git/pypprof/pypprof/net_http.py", line 20 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 783 in exec_module
  File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "/home/guillaumenoale/git/pypprof/test/test_net_http.py", line 8 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 783 in exec_module
  File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "/usr/lib/python3.8/unittest/loader.py", line 154 in loadTestsFromName
  File "/usr/lib/python3.8/unittest/loader.py", line 220 in <listcomp>
  File "/usr/lib/python3.8/unittest/loader.py", line 220 in loadTestsFromNames
  File "/usr/lib/python3.8/unittest/main.py", line 158 in createTests
  File "/usr/lib/python3.8/unittest/main.py", line 147 in parseArgs
  File "/usr/lib/python3.8/unittest/main.py", line 100 in __init__
  File "/usr/lib/python3.8/unittest/__main__.py", line 18 in <module>
  File "/usr/lib/python3.8/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.8/runpy.py", line 194 in _run_module_as_main
Segmentation fault (core dumped)

I tried some import independently, I am able to import this module "/home/guillaumenoale/git/pypprof/lib/python3.8/site-packages/google/protobuf/descriptor.py" and this package from zprofile.cpu_profiler import CPUProfiler
The specific sequence of import in net_http.py until the one below (line 20 included) seems to cause the issue

[timpalpant]

Thanks for reporting. There appears to be some incompatibility with the protobuf C++ module. You can see this by running with PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python, which will force protobuf to use the Python implementation instead.

Possibly related:

• segmentation fault for protos with optional fields protocolbuffers/protobuf#7801
• [Feature Request] Allow custom namespace to avoid protobuf version/linking conflicts protocolbuffers/protobuf#4004
• Protobuf C++ Python extension segfaults protocolbuffers/protobuf#5251
• https://bugs.python.org/issue37434

[timpalpant]

There seems to be an issue in mprofile. It reproduces easily with the pre-built wheels, but goes away if the module is rebuilt on a more recent Ubuntu. I do not yet understand the root cause but have updated it to build with Ubuntu 16.04 instead of Ubuntu 14.04 (timpalpant/mprofile@f690146)

[tmc]

I'm getting failures on python:3.7-slim images which is this debian release:

# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"