Search code examples
pythonsubprocesskill-process

How to wait after process kill until the file is unlocked on Windows using Python?

Problem description

I have a unittest that calls an external program and does some testing. After that, the external process is killed, and the test tries to clean up files the external program created. However, if I just call unlink() directly after the kill() command on Windows 10, I get:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'my.log'

If I time.sleep(4) before calling unlink() everything works as expected. The 4 was chosen arbitrarily, other times also work.

MCVE

This MCVE has two files. A server.py that simply locks a logging file and a test_client.py that calls the server, kills it and finally tries to remove the logging file.

test_client.py

import pathlib
import subprocess
import sys
import time

# Create test folder
server_path = pathlib.Path('.')
server_path.mkdir(exist_ok=True)

server_file = pathlib.Path('server.py').resolve()

# Start server in test folder
proc = subprocess.Popen([sys.executable, str(server_file)], cwd=server_path)

time.sleep(4)

# Kill server
proc.kill()

# Activate the following line to avoid the PermissionError: [WinError 32] ...
#time.sleep(4)

# Clean up
pathlib.Path('my.log').unlink()

server.py

import time
import logging

logging.basicConfig(
    filename='my.log',
    level=logging.DEBUG)

logging.info('I just started my work')

while True:
    time.sleep(1)

Question

  • Is there a way to wait for the kill to be finished?
  • Bonus if code is not platform specific
Solution

  • The correct way is 

    # Kill server
proc.kill()

# Activate the following line to avoid the PermissionError: [WinError 32] ...
proc.communicate()

# Clean up
pathlib.Path('my.log').unlink()

    The reason why it behaves like this require some documentation.

    As stated in Python Official Documentation,

    Popen.kill() Kills the child. On Posix OSs the function sends SIGKILL to the child. On Windows kill() is an alias for terminate().

    Popen.terminate() Stop the child. On Posix OSs the method sends SIGTERM to the child. On Windows the Win32 API function TerminateProcess() is called to stop the child.

    In windows, as stated, it is calling TerminateProcess function on Win32 API. While, it is clearly stated that

    TerminateProcess is asynchronous; it initiates termination and returns immediately. If you need to be sure the process has terminated, call the WaitForSingleObject function with a handle to the process.

    Therefore, kill() is just simply "Ask for stop". Not stopped yet. Thus, you need a method for 'waiting' the process end.

    communicate() is designed for this purpose too, as still stated in doc

    Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.