Search code examples
pythontestingpytestpython-unittestnose

Python: How to tests a program that uses fork?

I am writing a program that uses several instances of the same code in production. Instances then communicate over network, but that is irrelevant here.

During development, I fork the initial process multiple times to get a complete setup of instances, which is very convenient.

However, trying to write integration tests on this gives me problems: The test framework (unittest/nosetests/py.test) is of course also forked, so that I end up with multiple copies test framework, providing garbled results as output of each child's unittest overlaps.

A simplified example:

  • program is chat client, connects to another client via some network connection (say, redis pubsub)
  • program exposes an API to post and read messages
  • I want to fork off two program instances that connect to each other
  • tests should use API to make sure that the clients work by sending/receiving messages to/from each other
  • main process supervises the child processes for errors and runs the test suite

To achieve this, I'd need to "unload" or stop the test frameworks from the child processes. How can I do this?

I could fix this by invoking the test suite only after forking off the children, but I need to re-start all children for every test.

Solution

  • Forking a process in this context is a big hassle, as the child processes inherit the full stack of the parent, including the running instance of the test framework.

    There are essentially two options to solve this:

    Number one is to fork before initializing the test framework, and then access the child processes from the tests. This works well if child processes are long-lived, but proves to be a hassle if child processes are to be started per test, maybe even with different test fixtures/scenarios.

    Number two is to forgo forking all together by simply starting the processes by using subprocess / Popen. I found a pytest plugin called xprocess which helps you with this task. It didn't provide exactly what I needed, but it pointed me in the right direction.