How do I send a SIGUSR1 signal to tar, ran via subprocess in Python?

Question

I'm executing a tar process with the subprocess module and I discovered the ability to use signals to get progress information out of it (send to stderr).

$ tar -xpf archive.tar --totals=SIGUSR1 ./blah
$ pkill -SIGUSR1 tar  # separate terminal, session etc.

Unfortunately, I am unable to replicate this sequence successfully in Python.

import os
import subprocess
import signal
import time
import sys

# Define the command to execute
command = ["tar", "-xpf", sys.argv[2], "-C", sys.argv[1], "--totals=SIGUSR1"]

# Start the subprocess
print(' '.join(command))
process = subprocess.Popen(command, preexec_fn=os.setsid, stderr=subprocess.PIPE)

try:
    while True:
        # Ping the subprocess with SIGUSR1 signal
        # NOTWORK: process.send_signal(signal.SIGUSR1)
        # NOTWORK: os.killpg(os.getpgid(process.pid), signal.SIGUSR1)
        subprocess.Popen(["kill", "-SIGUSR1", str(process.pid)])

        print(process.stderr.readline().decode("utf-8").strip())
        # print(process.stdout.readline().decode("utf-8").strip())

        # Wait for a specified interval
        time.sleep(1.9)  # Adjust the interval as needed

except KeyboardInterrupt:
    # Handle Ctrl+C to gracefully terminate the script
    process.terminate()

# Wait for the subprocess to complete
process.wait()

The only style that works is opening a kill process with Popen like a cave man.

I noticed that after using os.kill or Popen.send_signal, the .poll() method returns -10. Seems like it dies after receiving the signal for some reason?

Answer 1

While Python is normally pretty slow, the signal-sending implementation is too fast for tar.

Before the process has even setup signal handlers, I end up sending the SIGUSR1 signal to it - and the default handling for this is to terminate the process.

On Linux, signal handlers can be inspected using the /proc/{pid}/status virtual filesystem.

Inside the /status virtual you can inspect the signal handlers. They are set as individual bits.

SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000384004
SigCgt: 0000000008013003  <--- This one
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 000001ffffffffff
CapAmb: 0000000000000000

Here's an implementation that reads, parses & checks whether the signal we are looking for is implemented (note: Linux only):

import sys
import signal
import subprocess
import time

if sys.platform == 'linux':
    def wait_until_ready(pid: int, sig: int, timeout: int = 1):
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            mask = 0
            with open('/proc/{}/status'.format(pid)) as f:
                for line in f.readlines():
                    if line[:7].lower() == 'sigcgt:':
                        mask = int(line[7:], 16)
                        break
            if mask & (1 << (sig - 1)):
                break
else:
    def wait_until_ready(pid: int, sig: int, timeout: int = 1):
        time.sleep(timeout)

cmd = ['tar', '-xpf', sys.argv[2], '-C', sys.argv[1], '--totals=SIGUSR1']
with subprocess.Popen(cmd, stderr=subprocess.PIPE, text=True) as p:
    try:
        wait_until_ready(p.pid, signal.SIGUSR1)
        while p.poll() is None:
            p.send_signal(signal.SIGUSR1)
            time.sleep(1)
            print(p.stderr.readline().strip())
    except KeyboardInterrupt:
        p.terminate()

This solution was presented by Eryk Sun in this discussion.