Search code examples
pythonimportscopemockingpatch

Why is the pytest mocker patching module methods globally (depending on the modules import)?


Problem:

For testing, I want to patch time.sleep in my main module (and assert its call), without patching it in submodules. However, importing time in my modules and patching time.sleep in my main module (mocker.patch("main.time.sleep")) mocks time.sleep in all submodules as well (Scenario A).
Concurrently, importing sleep from time and patching sleep in my main module (mocker.patch("main.sleep")) mocks sleep only in the main module (Scenario B, desired result).

Questions

Why is that? Why does the import make a difference? Why is time.sleep mocked globally in the first place? Shouldn't it be mocked only where it is looked up?

Thoughts

According to unittests Where-To-Patch-Article "patch() works by (temporarily) changing the object that a name points to with another one".
→ So the method sleep in time should be replaced for all modules in both scenarios?

Also, "you patch where an object is looked up, which is not necessarily the same place as where it is defined".
→ So maybe the sleep method is not replaced for all modules?

The article also states for a main module b:

  • if SomeClass is imported from module a and SomeClass() is called in main module bSomeClass is looked up in main module b

  • if module a is imported and a.SomeClass() is called in main module bSomeClass is looked up in module a

    → Is that the reason, why sleep is mocked globally in scenario A and locally in scenario B? But isn't sleep in both scenarios just a reference to the very same time.sleep method?

According to this discussion, importing (eg) time in different modules means sharing the time module. And patching a method of it in the main module patches it for all other modules as well. Whereas patching the whole time module in main would solely change the reference of time in main.
→ I am confused. Why so? When are we having references, when are we sharing modules, when are we mocking globally, when are we mocking locally and what is the logic behind all that?

Setup (Not working, Scenario A):

main.py:

import time
import module

def sleep_in_main():
    time.sleep(1)  # this is mocked
    module.sleep_in_module()

module.py:

import time

def sleep_in_module():
    time.sleep(1)  # this is also mocked

test.py:

import main

def test(mocker):
    sleep_mock = mocker.patch("main.time.sleep")
    main.sleep_in_main()
    assert sleep_mock.call_count == 1  # fails (is called twice)

Setup (Working, Scenario B):

main.py:

from time import sleep
import module

def sleep_in_main():
    sleep(1)  # this is mocked
    module.sleep_in_module()

module.py:

from time import sleep

def sleep_in_module():
    sleep(1)  # this is not mocked

test.py:

import main

def test(mocker):
    sleep_mock = mocker.patch("main.sleep")
    main.sleep_in_main()
    assert sleep_mock.call_count == 1  # passes

Solution

  • The reason for this behavior is how the python/mocking framework work.

    When you do import time you essentially say: import time as time in your module, which is like defining an attribute: time = reference to time module.

    Later, when you define sleep_in_main method, you add another attribute sleep_in_main = reference to method body.

    If however you do from time import sleep you essentially define sleep = reference to sleep from time module.

    Now, all mocking does is replace the attribute, so if you do mocker.patch("main.sleep") it means sleep = MagicMock() for your main module. It doesn't affect the other modules since it touches the sleep attribute of the main module.

    If you do mocker.patch("main.time.sleep") the mock looks up main.time, which is a reference to the time module, then it accesses the sleep attribute from that reference, but that sleep is defined in the time module, so it does sleep = MagicMock() in the time module itself, which affects all the modules importing it.

    This reference/attribute game is similar to what happens to function parameters. Say this is your code:

    def f1(p):
        p.append(3)
    
    def f2(p):
        p = [5]
    
    a = [1, 2]
    
    f1(a)
    f2(a)
    

    The value of a would be [1, 2, 3], since the f1 is referring a and changing it, while f2 repoints the variable p to an entirely different list.

    Hope this makes sense.