Resume process after signal

Question

I have implemented a child process in Python that is supposed to do the following steps: (1) Initialize process, (2) Wait for 'start' signal from parent, (3) Process some data and write it to shared memory, (4) Send 'finish' signal to parent and (5) Goto (2).

This is basically my setup:

import signal
import os

signal_received = False

def signal_handler(signo, _):
    if signo == signal.SIGUSR1:
        signal_received = True

def main():
    global signal_received

    signal.signal(signal.SIGUSR1, signal_handler)

    while True:
        # solution attempt:
        # signal.sigsuspend(...)

        while not signal_received:
            signal.pause() 

        signal_received = False
        process()

        # solution attempt:
        # signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGUSR1])

        os.kill(os.getppid(), signal.SIGUSR1)

The problem I'm running into is described here: If the signal from the parent is received before signal.pause() has been reached (i.e. in the short time frame where the while loop checks signal_received), the signal is lost and the child process halts.

One solution in C would be to use a combination of sigmask and sigsuspend, but Python does not offer a sigsuspend. Is there a solution for this problem in Python?

Edit:

I tried implementing the self-pipe trick, but the child process still halts after a (random) amount of time. As far as I can tell, this depends on whether os.read(rfh, 1) already started executing when the parent sends the SIGUSR1 signal. Although the signal is sent by the parent, the child does not execute the signal_handler.

import signal
import os

rfh, wfh = os.pipe()

def signal_handler(signo, _):
    if signo == signal.SIGUSR1:
        os.write(wfh, b"\x00")

def main():
    signal.signal(signal.SIGUSR1, signal_handler)

    while True:
        os.kill(os.getppid(), signal.SIGUSR1)
        os.read(rfh, 1)
        # foo() (some processing)

if __name__ == "__main__":
    main()

#include <unistd.h>
#include <signal.h>
#include <iostream>

static volatile bool signal_received = false;

void handle(int signo, siginfo_t *, void *)
{
    if (signo == SIGUSR1)
        signal_received = true;
}

int main()
{
    // Setup signal handler
    struct sigaction act = {0};
    act.sa_flags = SA_SIGINFO | SA_RESTART;
    act.sa_sigaction = &handle;
    sigaction(SIGUSR1, &act, NULL);

    pid_t pid = fork();
    if (pid == 0)
    {
        char path[] = "/usr/bin/python3";
        char *const argv[] = {"python3", "main.py", NULL};
        char *const env[] = {NULL};

        execve(path, argv, env);
    }

    sigset_t mask, oldmask;
    sigemptyset(&mask);
    sigaddset(&mask, SIGUSR1);

    while (true)
    {
        static int iteration = 0;
        std::cout << iteration++ << std::endl;

        sigprocmask(SIG_BLOCK, &mask, &oldmask);

        while (!signal_received)
            sigsuspend(&oldmask);

        signal_received = false;

        sigprocmask(SIG_UNBLOCK, &mask, NULL);

        kill(pid, SIGUSR1);
    }

    return 0;
}

Edit 2:

I have found a solution that works on MacOS and Linux (MacOS has no os.pipe2):

import signal
import os
import fcntl

rfh, wfh = os.pipe()
fcntl.fcntl(wfh, fcntl.F_SETFL, os.O_NONBLOCK)

def main():
    signal.signal(signal.SIGUSR1, lambda *_: None)
    signal.set_wakeup_fd(wfh)

    while True:
        os.kill(os.getppid(), signal.SIGUSR1)
        os.read(rfh, 1)

if __name__ == "__main__":
    main()

Answer 1

Yes there is a solution, it is called the self-pipe trick.

1. Create a pipe (docs: os.pipe), a pipe is a pair of connected file descriptors, one for reading and one for writing
2. Make the signal handler write anything (one byte is OK) to the writer-end of the pipe. Update from comments: a non-blocking write is recommended.
3. To wait for the signal, read from the pipe's reader-end. If you want just to check, use a non-blocking read or select.select.

Here is a tiny demo:

import os
import signal

rfd, wfd = os.pipe()

def handler(*args):
    os.write(wfd, b"\x00")

signal.signal(signal.SIGUSR1, handler)
print(f"waiting, send SIGUSR1 to PID {os.getpid()}")
os.read(rfd, 1)
print("got signal")

Note that since Python 3.4, I/O (here: os.read) interrupted by a signal is automatically restarted.

---

Important update:

Under certain circumstances and for reasons not clear yet, the Python process sometimes receives the signal but does not call the corresponding handler. A similar program written in C works fine. I think it might be a bug.

Here is an alterntive that does not have this problem. It is still based on the "self-pipe trick", but uses signal.set_wakeup_fd() from the standard library. If set, it is active for every signal, so be careful when catching several signal types.

import fcntl
import os
import signal

sigusr1 = False

def handler(signo, _): 
    global sigusr1
    if signo == signal.SIGUSR1:
        sigusr1 = True

rfd, wfd = os.pipe()
# make writes non-blocking
flags = fcntl.fcntl(wfd, fcntl.F_GETFL)
fcntl.fcntl(wfd, fcntl.F_SETFL, flags | os.O_NONBLOCK)

signal.set_wakeup_fd(wfd)
signal.signal(signal.SIGUSR1, handler)

print(f"waiting, send SIGUSR1 to PID {os.getpid()} or ctrl-C to abort")
while True:
    while not sigusr1:
        os.read(rfd, 1)
    sigusr1 = False
    print("got signal")

Note: the signal number can be also read from the pipe, but some handler for the signal itself must be set, because the default is to terminate the process.