Cancellation points in signal handlers?

What happens if a program calls a function which is a cancellation point from a signal handler? There are a number of functions which POSIX specifies as both async-signal-safe and cancellation points. If a signal handler calls such a function and cancellation is acted upon, the result is quite similar to what would happen if the thread had enabled asynchronous cancellation - actually much worse, because all the cancellation cleanup handlers, which are probably not async-signal-safe, would be called from a signal-handler context.

What does POSIX actually specify in this case, and what do implementations actually do? I can't find any language in POSIX that would forbid cancellation points in signal handlers from being acted upon, nor any such protection in the glibc/nptl source.

Solution

I'm not aware that POSIX even dares to mention this topic, but I haven't done an exhaustive search.

Some brief experimentation with a gcc/nptl system reveals that, as I suspected and I think you did too, there is no such protection in NPTL - the cancellation handlers do indeed get called, from within the signal handler context.

The program below (apologies for the hackiness etc) displays the following output:

Signal handler called
Sent cancellation
Cleanup called
In sighandler

... indicating that:

the signal handler got called
the other thread then called pthread_cancel()
the cancellation handler then got called, without the signal handler completing

Here's the program:

#include <stdio.h>
#include <pthread.h>
#include <signal.h>
#include <string.h>
#include <unistd.h>
#include <assert.h>

pthread_t mainthread;

int in_sighandler = 0;

void
cleanup (void *arg)
{
    write(1, "Cleanup called\n", strlen("Cleanup called\n"));
    if (in_sighandler) {
        write(1, "In sighandler\n", strlen("In sighandler\n"));
    } else {
        write(1, "Not in sighandler\n", strlen("In sighandler\n"));
    }
}


void
sighandler (int sig, siginfo_t *siginfo, void *arg)
{
    in_sighandler = 1;
    write(1,"Signal handler called\n", strlen("Signal handler called\n"));  // write() is a CP
    usleep(3000000); // usleep() is a CP; not strictly async-signal-safe but happens to be so in Linux
    write(1, "Signal handler exit\n", strlen("Signal handler exit\n"));
    in_sighandler = 0;
}

void *
thread (void *arg)
{
    sleep(1);
    pthread_kill(mainthread, SIGUSR1);
    usleep(500000);
    pthread_cancel(mainthread);
    printf("Sent cancellation\n");
    return (NULL);
}

int
main (int argc, char **argv)
{
    int rc;
    struct sigaction sa;
    pthread_t threadid;

    mainthread = pthread_self();

    // Set up a signal handler to test its cancellation properties
    sa.sa_sigaction = &sighandler;
    sigemptyset(&sa.sa_mask);
    sa.sa_flags = SA_SIGINFO;
    rc = sigaction(SIGUSR1, &sa, NULL);
    assert(rc == 0);

    // Set up a thread to send us signals and cancel us
    rc = pthread_create(&threadid, NULL, &thread, NULL);
    assert(rc == 0);

    // Set up cleanup handlers and loop forever
    pthread_cleanup_push(&cleanup, NULL);
    while (1) {
        sleep(60);
    }
    pthread_cleanup_pop(0);
    return (0);
}