Flask application randomly refuses connections when testing

Question

I have an API written in Flask and am testing the endpoints with nosetests using requests to send a request to the API. During the tests, I randomly get an error

ConnectionError: HTTPConnectionPool(host='localhost', port=5555): Max retries exceeded with url: /api (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fe4e794fd50>: Failed to establish a new connection: [Errno 111] Connection refused',))

This error only seems to happen when running tests and randomly affects anywhere between none and all of the tests. All of my tests are being run from one subclass of unittests.TestCase:

class WebServerTests(unittest.TestCase):
    # Args to run web server with
    server_args = {'port': WEB_SERVER_PORT, 'debug': True}

    # Process to run web server
    server_process = multiprocessing.Process(
        target=les.web_server.run_server, kwargs=server_args)

    @classmethod
    def setup_class(cls):
        """
        Set up testing
        """
        # Start server
        cls.server_process.start()

    @classmethod
    def teardown_class(cls):
        """
        Clean up after testing
        """
        # Kill server
        cls.server_process.terminate()
        cls.server_process.join()

    def test_api_info(self):
        """
        Tests /api route that gives information about API
        """
        # Test to make sure the web service returns the expected output, which at
        # the moment is just the version of the API
        url = get_endpoint_url('api')
        response = requests.get(url)
        assert response.status_code == 200, 'Status Code: {:d}'.format(
            response.status_code)
        assert response.json() == {
            'version': module.__version__}, 'Response: {:s}'.format(response.json())

Everything is happening on localhost and the server is listening on 127.0.0.1. My guess would be that too many requests are being sent to the server and some are being refused, but I'm not seeing anything like that in the debug logs. I had also thought that it may be an issue with the server process not being up before the requests were being made, but the issue persists with a sleep after starting the server process. Another attempt was to let requests attempt retrying the connection by setting requests.adapters.DEFAULT_RETRIES. That didn't work either.

I've tried running the tests on two machines both normally and in docker containers and the issue seems to occur regardless of the platform on which they are run.

Any ideas of what may be causing this and what could be done to fix it?

Answer 1

It turns out that my problem was indeed an issue with the server not having enough time to start up, so the tests would be running before it could respond to tests. I thought I had tried to fix this with a sleep, but had accidentally placed it after creating the process instead of after starting the process. In the end, changing

cls.server_process.start()

to

cls.server_process.start()
time.sleep(1)

fixed the issue.