Fix Ctrl-C segfaults in sage.libs.gap #40613

orlitzky · 2025-08-17T14:33:06Z

Replace cysignals' sig_on() and sig_off() with custom gap_sig_on() and gap_sig_off() wrappers that use GAP's own SIGINT handling. This fixes an uncommon, but ultimately reproducible segfault when Ctrl-C is used to interrupt a GAP computation.

In concert with #40594 this has allowed sage/libs/gap/element.pyx to pass many thousands of test iterations.

Fixes

Use InterruptExecStat for GAP interrupt #40598

Dependencies

Fix segmentation fault in libgap function call #40594

We plan to install InterruptExecStat (gap/stats.h) as the SIGINT and SIGALRM handler while GAP code is running, since that seems to work better than mixing cysignals with GAP_Enter/GAP_Leave has.

Add two new pre/post-GAP functions to enable/disable GAP's own SIGINT (Ctrl-C) handler. These avoid the setjmp/longjmp issues we've encountered with the sig_on() and sig_off() from cysignals.

Replace the sig_on() and sig_off() wrappers from cysignals with the new custom gap_sig_on() and gap_sig_off(). We've lost the sig_on() and sig_off() around GAP_initialize() because I don't think we can trust GAP's own signal handler until after we've initialized GAP.

Replace the sig_on() and sig_off() wrappers from cysignals with the new custom gap_sig_on() and gap_sig_off(). This fixes the segfault that occurs after repeated testing of _pow_.

We are no longer using cysignals for this; instead we have our own gap_sig_on() and gap_sig_off() functions.

The way we handle signals in libgap is now unusual, so a few paragraphs have been added to the libgap module docstring explaining how and why it is unusual.

We've found that GAP's own SIGINT handler is less crashy than if we mix GAP_Enter/GAP_Leave with cysignals' sig_on and sig_off. We've installed that same handler for SIGALRM, but the code that catches it can't tell the difference and converts them both in to KeyboardInterrupt. To retain the ability to doctest this stuff, we have to catch those KeyboardInterrupts and poke at them to see if they arose from GAP.

This function used cysignals for sig_error(), and we have opted not to mix cysignals code with libgap code. It has in any case been replaced by plain GAP_Enter calls.

src/sage/libs/gap/libgap.pyx

src/sage/libs/gap/util.pyx

src/sage/libs/gap/libgap.pyx

user202729 · 2025-08-17T14:52:58Z

src/sage/libs/gap/element.pyx

+                # Ctrl-C
+                raise KeyboardInterrupt from e
+            else:
+                raise


this is some really bad duplication of code. I suggest

fold GAP_Enter and GAP_Leave into gap_sig_on and gap_sig_off respectively

instead of setting the interrupt handler directly to InterruptExecStat, make custom wrapper function (or hook into cysignals), the custom wrapper function sets a global variable that determine which kind of signal is being raised, then call InterruptExecStat

the global variable above need to be reset at appropriate places

modify error_handler to, instead of indiscriminately raise GAPError, raise the correct exception by reading the global variable above

Yeah it's pretty bad but I was trying to keep it as simple as possible for the first iteration. First get it to not crash, then make it pretty.

I don't think we can use a global for the signal type because signals are asynchronous. There's no reason why a SIGINT can't be triggered while handling a SIGALRM, or vice-versa. It's not very likely, but if there's a lesson here it's that we shouldn't count on unlikely things not happening.

We may be able to combine GAP_Enter and gap_sig_on(), though.

I don't think we can use a global for the signal type because signals are asynchronous. There's no reason why a SIGINT can't be triggered while handling a SIGALRM, or vice-versa. It's not very likely, but if there's a lesson here it's that we shouldn't count on unlikely things not happening.

you're right but… I think currently the signal handler is calling PyErr_Restore() or something anyway, and that one isn't reentrant either.

Searching up a bit https://stackoverflow.com/a/3127697/, that's indeed a possibility. But doesn't the same thing happen for cysignals as well?

We may be able to combine GAP_Enter and gap_sig_on(), though.

Even this immediately leads to a segault.

user202729 · 2025-08-17T14:54:47Z

src/sage/doctest/util.py

+            e.__traceback__ = None
+            alarm_raised = True
+        else:
+            raise


if the logic below (raise AlarmInterrupt instead of KeyboardInterrupt) are correctly implemented, this change wouldn't be necessary.

I've further simplified this down to a one-line change, using the fact that AlarmInterrupt is a subclass of KeyboardInterrupt. Instead of catching AlarmInterrupt, we can catch KeyboardInterrupt, and then check,

is it an AlarmInterrupt?

does it have "user interrupt" as the message?

I prefer this to hacking AlarmInterrupt into the GAP code at this point because sage.libs.gap is otherwise free of cysignals. To raise the AlarmInterrupt from sage.libs.gap we'd have to,

Add cysignals back as a dep in meson.build

Import AlarmInterrupt

Add the global last_signal variable

Set last_signal whenever a handler is called

Check last_signal when raising a GAPError and convert it to the right thing

Since this is all for the benefit of one doctest method, it just seems easier to add the one line in that doctest method?

user202729 · 2025-08-17T15:10:37Z

I find the current implementation (before this pull request) questionable also—one shouldn't be calling any Python code between GAP_Enter…GAP_Leave, and if there's no Python code, try…finally isn't necessary.

If I understood correctly, the way the current exception handler works is that it just sets PyErr_Restore and hope that Python checks the exception quickly enough.

One implementation idea:

whenever you need to call GAP API, you only need to

gap_sig_on()
gap_check_exceptions()
<GAP code here, must not contain any Python code, therefore cannot raise any exception>
gap_sig_off()

where you do something like

int gap_enter_ok; // global variable internal to that file
#define gap_sig_on() gap_set_signal_handlers(); gap_enter_ok = GAP_Enter()
#define gap_check_exceptions() if (!gap_enter_ok) { GAP_Leave(); raise_appropriate_exception_type() }
#define gap_sig_off() GAP_Leave(); gap_reset_signal_handler()

github-actions · 2025-08-17T15:23:41Z

Documentation preview for this PR (built with commit c1adeb9; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

orlitzky · 2025-08-17T23:34:04Z

I find the current implementation (before this pull request) questionable also—one shouldn't be calling any Python code between GAP_Enter…GAP_Leave, and if there's no Python code, try…finally isn't necessary.

If I understood correctly, the way the current exception handler works is that it just sets PyErr_Restore and hope that Python checks the exception quickly enough.

I think the error handler returns control to GAP, though, which eventually jumps back to GAP_Enter. If I move the GAP_Enter outside of try/except, we won't catch the GAPError that is raised. I left comments on the GAP_Enter lines because this surprised me at first.

user202729 · 2025-08-17T23:38:33Z

I think the error handler returns control to GAP, though, which eventually jumps back to GAP_Enter. If I move the GAP_Enter outside of try/except, we won't catch the GAPError that is raised. I left comments on the GAP_Enter lines because this surprised me at first.

In my understanding, the way it works is the following. GAP_Enter is just a C macro, defined by

cdef extern from "gap/libgap-api.h" nogil:
    cdef int GAP_Enter() except 0

that by itself cannot raise an exception. But PyErr_Restore() can, and the exception is raised the next time Python checks for pending exceptions.

But the problem I see is that this looks dangerous, since to me there doesn't seem to be any guarantee it will be checked right after GAP_Enter call.

Anyway, need to double check.

orlitzky · 2025-08-18T02:29:47Z

int gap_enter_ok; // global variable internal to that file
#define gap_sig_on() gap_set_signal_handlers(); gap_enter_ok = GAP_Enter()
#define gap_check_exceptions() if (!gap_enter_ok) { GAP_Leave(); raise_appropriate_exception_type() }
#define gap_sig_off() GAP_Leave(); gap_reset_signal_handler()

I'm trying a simpler version of this to start, with just gap_set_signal_handlers(), gap_sig_on(), and gap_sig_off() in an .h file, and it's already crashing as soon as I sent Ctrl-C or an alarm. I'm giving up for the night. I'm getting nowhere.

user202729 · 2025-08-18T11:35:26Z

I guess if you don't understand the segmentation fault very well it would be unfair. Not sure if the rare segmentation fault is worth the code duplication though.

Anyway what is your current implementation?

(Note that the GAP_Enter cannot be nested/hidden in anything, which is why I'm forced to separate the gap_check_exceptions from gap_sig_on and make them macros instead of inline functions.)

By the way, for quick hacking, you can use inline raw C embedded in pxd/pyx file with extern from *, instead of separate .h file.

If we're going to raise a GAPError that resulted from a SIGINT or SIGALRM, we might as well turn it in to a KeyboardInterrupt right away.

We are no longer retaining the __cause__ of KeyboardInterrupts that arise from GAP, but the error message is still there, so we can use that to filter these out.

We are now converting "GAPError: Error, user interrupt" into a KeyboardInterrupt in our libgap error handler, so we don't have to do it at every call site.

src/sage/libs/gap/util.pyx

src/sage/libs/gap/gap_includes.pxd

user202729 · 2025-08-22T00:30:51Z

I'm not sure why this function is using sig_block() and sig_unblock() in the first place -- you're probably not supposed to run them in a loop.

to avoid interrupting in the middle of the user callback function which is likely to corrupt the Python state.

I've seen it take 3s

okay, that's surprising. But then maybe that can happen if x is really close to a multiple of π, or something like the table maker's dilemma, or because trigonometric functions are just slow.

user202729 · 2025-08-23T06:43:03Z

minor documentation formatting issue in rendered HTML https://doc-pr-40613--sagemath.netlify.app/html/en/reference/libs/sage/libs/gap/libgap#using-the-gap-c-library-from-cython (the indentation is interpreted as quotation, and some code blocks aren't interpreted as such because of the lack of ::), but no big deal. You can re-set positive review afterwards.

I may put in some other improvements to remove the string check some day (just store the signum received by the signal handler to some global variable, then read it from the error handler, instead of detecting the user interrupt string).

(and maybe expose some way to customize the class used for alarm interrupt, in order to raise AlarmInterrupt, for consistency with the rest of the code base.)

user202729 · 2025-08-23T06:48:26Z

actually I think you want to rebase it on top of #40594 (instead of cherry-pick the first 2 commits as in the current situation) to avoid polluting the commit history.

These never should have been indented in the first place; the whole block is being rendered as a quote.

This is being rendered as a quote.

orlitzky · 2025-08-23T14:46:20Z

actually I think you want to rebase it on top of #40594 (instead of cherry-pick the first 2 commits as in the current situation) to avoid polluting the commit history.

git merge and git rebase will skip the two redundant commits so long as your branch is merged first. I added your PR as a dependency of this one in the description to ensure that the order is correct.

orlitzky · 2025-08-23T16:54:21Z

Docs should be fixed now.

user202729 · 2025-08-25T14:22:18Z

I added your PR as a dependency of this one in the description to ensure that the order is correct.

huh, does @vbraun actually read these? I thought they're for the benefit of reviewers.

orlitzky · 2025-08-25T14:36:32Z

huh, does @vbraun actually read these? I thought they're for the benefit of reviewers.

I hope so, we don't have any other way to express dependencies between PRs and this is how we imported them from trac.

user202729 · 2025-08-29T00:15:05Z

bad news, segmentation fault again here?

https://github.com/sagemath/sage/actions/runs/17294515300

orlitzky · 2025-08-29T18:57:57Z

bad news, segmentation fault again here?

https://github.com/sagemath/sage/actions/runs/17294515300

I'm not sure, I can't reproduce that one.

user202729 · 2025-08-30T02:46:01Z

I guess the cherry-picked commits get duplicated anyway (if you git log you'll see both e0b0c06 and 35e479a being in the commit history), but nothing too bad is caused by it.

orlitzky · 2025-08-30T12:40:18Z

I guess the cherry-picked commits get duplicated anyway (if you git log you'll see both e0b0c06 and 35e479a being in the commit history), but nothing too bad is caused by it.

Ok, you were right. I tested and the history was clean even with a merge commit, so I don't know what's happening in the sage repo but it would be much better if the commit history were sane. As it stands git log -p will show you changes that could not possibly have happened, even with --no-merges or --first-parent.

orlitzky · 2025-09-03T01:52:44Z

bad news, segmentation fault again here?

https://github.com/sagemath/sage/actions/runs/17294515300

I guess these are not new: #37295

sagemathgh-40727: Explicitly check signum in GAP error handler Follow-up to sagemath#40613. Instead of checking for the string `user interrupt` (which might change between GAP versions, or if there's some unforeseen way the string might be sneaked in), we store the signum from the signal handler then check it in the GAP error handler. Also optionally use `AlarmInterrupt` instead of `KeyboardInterrupt` if cysignals is available. ### 📝 Checklist  - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies    URL: sagemath#40727 Reported by: user202729 Reviewer(s): Michael Orlitzky, user202729

user202729 and others added 10 commits August 16, 2025 22:19

Fix segmentation fault in libgap function call

e0b0c06

Code modification to avoid calling GapElement.__dealloc__ too early

4d1f19f

src/sage/libs/gap/gap_includes.pxd: add InterruptExecStat

c6b76b6

We plan to install InterruptExecStat (gap/stats.h) as the SIGINT and SIGALRM handler while GAP code is running, since that seems to work better than mixing cysignals with GAP_Enter/GAP_Leave has.

src/sage/libs/gap/util.pyx: add gap_sig_on / gap_sig_off

f9c186d

Add two new pre/post-GAP functions to enable/disable GAP's own SIGINT (Ctrl-C) handler. These avoid the setjmp/longjmp issues we've encountered with the sig_on() and sig_off() from cysignals.

src/sage/libs/gap/element.pyx: replace sig_on / sig_off

fc16eae

Replace the sig_on() and sig_off() wrappers from cysignals with the new custom gap_sig_on() and gap_sig_off(). This fixes the segfault that occurs after repeated testing of _pow_.

src/sage/libs/gap/meson.build: remove cysignals dependency

a887924

We are no longer using cysignals for this; instead we have our own gap_sig_on() and gap_sig_off() functions.

src/sage/libs/gap/libgap.pyx: update signal handling docs

a219ac9

The way we handle signals in libgap is now unusual, so a few paragraphs have been added to the libgap module docstring explaining how and why it is unusual.

src/sage/libs/gap/gap_includes.pxd: drop sig_GAP_Enter()

e1e8d62

This function used cysignals for sig_error(), and we have opted not to mix cysignals code with libgap code. It has in any case been replaced by plain GAP_Enter calls.

orlitzky requested review from user202729 and dimpase August 17, 2025 14:33

github-actions bot added the s: needs review label Aug 17, 2025

user202729 reviewed Aug 17, 2025

View reviewed changes

src/sage/libs/gap/libgap.pyx Outdated Show resolved Hide resolved

user202729 reviewed Aug 17, 2025

View reviewed changes

src/sage/libs/gap/util.pyx Show resolved Hide resolved

user202729 reviewed Aug 17, 2025

View reviewed changes

src/sage/libs/gap/libgap.pyx Outdated Show resolved Hide resolved

user202729 reviewed Aug 17, 2025

View reviewed changes

src/sage/libs/gap/libgap.pyx: rewrite signal handling explanation

2ca060f

orlitzky added 3 commits August 18, 2025 12:10

src/sage/libs/gap/util.pyx: do KeyboardInterrupt conversion earlier

7c48f94

If we're going to raise a GAPError that resulted from a SIGINT or SIGALRM, we might as well turn it in to a KeyboardInterrupt right away.

src/sage/doctest/util.py: update GAP workaround

fb458ad

We are no longer retaining the __cause__ of KeyboardInterrupts that arise from GAP, but the error message is still there, so we can use that to filter these out.

src/sage/libs/gap: remove redundant KeyboardInterrupt hacks

51c81b0

We are now converting "GAPError: Error, user interrupt" into a KeyboardInterrupt in our libgap error handler, so we don't have to do it at every call site.

user202729 reviewed Aug 18, 2025

View reviewed changes

src/sage/libs/gap/util.pyx Outdated Show resolved Hide resolved

user202729 reviewed Aug 18, 2025

View reviewed changes

src/sage/libs/gap/gap_includes.pxd Show resolved Hide resolved

user202729 mentioned this pull request Aug 22, 2025

Fix segmentation fault in libgap function call #40594

Merged

5 tasks

user202729 added s: positive review and removed s: needs review labels Aug 23, 2025

orlitzky added 2 commits August 23, 2025 10:37

src/sage/libs/gap/libgap.pyx: unindent the "Using the .. library" docs

6e74192

These never should have been indented in the first place; the whole block is being rendered as a quote.

src/sage/libs/gap/libgap.pyx: unindent the AUTHORS block

47826c2

This is being rendered as a quote.

github-actions bot added s: needs review and removed s: positive review labels Aug 23, 2025

src/sage/libs/gap/libgap.pyx: backticks for libgap-api.h

c1adeb9

orlitzky added s: positive review and removed s: needs review labels Aug 23, 2025

dimpase approved these changes Aug 25, 2025

View reviewed changes

vbraun merged commit b2392aa into sagemath:develop Aug 27, 2025
26 of 29 checks passed

github-actions bot removed the s: positive review label Aug 27, 2025

This was referenced Aug 30, 2025

Explicitly check signum in GAP error handler #40727

Merged

Fix another libgap segmentation fault #40728

Draft

user202729 mentioned this pull request Sep 13, 2025

Rerun plural and singular/function on failure #40814

Merged

5 tasks

Uh oh!

Fix Ctrl-C segfaults in sage.libs.gap #40613

Fix Ctrl-C segfaults in sage.libs.gap #40613

Uh oh!

Conversation

orlitzky commented Aug 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes

Dependencies

Uh oh!

Uh oh!

Uh oh!

Uh oh!

user202729 Aug 17, 2025

Choose a reason for hiding this comment

Uh oh!

orlitzky Aug 17, 2025

Choose a reason for hiding this comment

Uh oh!

user202729 Aug 17, 2025

Choose a reason for hiding this comment

Uh oh!

orlitzky Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

user202729 Aug 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

orlitzky Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

user202729 commented Aug 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orlitzky commented Aug 17, 2025

Uh oh!

user202729 commented Aug 17, 2025

Uh oh!

orlitzky commented Aug 18, 2025

Uh oh!

user202729 commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

user202729 commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

user202729 commented Aug 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

user202729 commented Aug 23, 2025

Uh oh!

orlitzky commented Aug 23, 2025

Uh oh!

orlitzky commented Aug 23, 2025

Uh oh!

user202729 commented Aug 25, 2025

Uh oh!

orlitzky commented Aug 25, 2025

Uh oh!

Uh oh!

user202729 commented Aug 29, 2025

Uh oh!

orlitzky commented Aug 29, 2025

Uh oh!

user202729 commented Aug 30, 2025

Uh oh!

orlitzky commented Aug 30, 2025

Uh oh!

orlitzky commented Sep 3, 2025

Uh oh!

Uh oh!

orlitzky commented Aug 17, 2025 •

edited

Loading

user202729 Aug 17, 2025 •

edited

Loading

user202729 commented Aug 17, 2025 •

edited

Loading

github-actions bot commented Aug 17, 2025 •

edited

Loading

user202729 commented Aug 18, 2025 •

edited

Loading

user202729 commented Aug 22, 2025 •

edited

Loading

user202729 commented Aug 23, 2025 •

edited

Loading