Mock - Change return value depending on Mock object

Question

I'm trying to mock some objects in unittest but I need to have a different boolean value returned depending on the Mock object being evaluated in an if statement.

The function I'm testing looks something like this:

def run_tests():
    failed_tests = []
    for test in get_tests():
       if validate_schema(test):
           #some logic
       else:
           failed_tests.append(test)
    return failed_tests

def validate_schema(test):
    if test == foo:
        return True
    else:
        return False

I want to do, for example, three iterations of the for loop, whereby the first two iterations validate_schema() returns False, and the third returns True.

My unit test looks like this, trying to achieve this:

import unittest

from unittest.mock import Mock, patch
from app.test_runner import run_tests

MOCK_TEST_CASE_ONE = Mock()
MOCK_TEST_CASE_TWO = Mock()
MOCK_TEST_CASE_THREE = Mock()

class TestRunTests(unittest.TestCase):
    @patch('app.test_extractor.get_tests')
    @patch('app.test_parser.validate_schema')
    def test_two_fail_one_pass_test_on_validation(self, mock_get_tests, mock_validate_schema):
        mock_get_unit_tests.return_value = [MOCK_TEST_CASE_ONE, MOCK_TEST_CASE_TWO, MOCK_TEST_CASE_THREE]
        validation_results = [False, False, True]
        mock_validate_schema.side_effect = validation_results
        self.assertEqual(run_tests(), [MOCK_TEST_CASE_ONE, MOCK_TEST_CASE_TWO])


if __name__ == "__main__":
    unittest.main()

However, when running the test, the test fails on mock_validate_schema.side_effect = validation_results as it returns the error TypeError: 'bool' object is not iterable.

I have tried following this similar example How to dynamically mock the result of a python function inside a for loop? but the difference there is foo is taking in a set list of items, whereas I'm trying to evaluate Mock objects, so I'm unsure how to evaluate the first two Mock objects to return False and third True.

Answer 1

After much researching and playing around, I figured out what was going wrong, thanks to this article I stumbled upon https://nedbatchelder.com/blog/202202/why_your_mock_still_doesnt_work.html

The order of your patching in your test function arguments needs to be back to front to how your decorators are called. Originally my test function args looked like this:

@patch('app.test_extractor.get_tests')
@patch('app.test_parser.validate_schema')
def test_two_fail_one_pass_test_on_validation(self, mock_get_tests, mock_validate_schema):

but the mock_get_tests and mock_validate_schema needs to be switched around as so:

@patch('app.test_extractor.get_tests')
@patch('app.test_parser.validate_schema')
def test_two_fail_one_pass_test_on_validation(self, mock_validate_schema, mock_get_tests):

The rest of the unit test was actually fine and now works, so it now looks like

import unittest

from unittest.mock import Mock, patch
from app.test_runner import run_tests

MOCK_TEST_CASE_ONE = Mock()
MOCK_TEST_CASE_TWO = Mock()
MOCK_TEST_CASE_THREE = Mock()

class TestRunTests(unittest.TestCase):
    @patch('app.test_extractor.get_tests')
    @patch('app.test_parser.validate_schema')
    def test_two_fail_one_pass_test_on_validation(self, mock_validate_schema, mock_get_tests):
        mock_get_unit_tests.return_value = [MOCK_TEST_CASE_ONE, MOCK_TEST_CASE_TWO, MOCK_TEST_CASE_THREE]
        validation_results = [False, False, True]
        mock_validate_schema.side_effect = validation_results
        self.assertEqual(run_tests(), [MOCK_TEST_CASE_ONE, MOCK_TEST_CASE_TWO])


if __name__ == "__main__":
    unittest.main()