Search code examples
matlab

Unit testing with file-wide variables, without classes


Suppose, for whatever reason, I refuse to use matlab.unittest.TestCase. I seek to reproduce the following Python functionality:

# `utils.py` is in same directory as the test file
# `test_wide_variable` gets used inside or outside of test functions
from utils import test_wide_variable

# gets used inside of test functions
file_wide_variable = [1, 2, 3]

def test_func1():
    y = (file_wide_variable, test_wide_variable)
    # ...

# ...

test_func1()
# test_func2()
test_func3()
# ...

Attempt

Define reset_test_params.m, with

global P
P = struct;

Then, the test file looks like

reset_test_params

P.file_wide_variable = [1, 2, 3];
P.test_wide_variable = load('test_wide_variables.m').test_wide_variable;

test_func1()
% test_func2()
test_func3()
% ...

function test_func1(~)
    global P
    y = {P.file_wide_variable, P.test_wide_variable};
    % ...

% ...

Question

Any notable downsides with this approach? In particular, concurrent file execution? With Python one can run multiple tests at once in same global namespace (AFAIK), so a global variable would be a no-no.


Solution

  • I just learned that you do use the builtin test harness runtests.. Other details were also not clear when I wrote the answer below the break. So here is my recommendation to avoid globals:

    All you need to do is start your file with function <name>, and end it with an end. This turns all the test functions inside into nested functions, as explained below. Now remove all mentions of the keyword global, and things should work as you expect them to work, using file-wide variables shared among the test functions in your file but not across files.

    I still strongly recommend not calling a script from within code, some of the dangers are described below. It is no effort whatsoever to turn that into a function that returns the constant data.


    Original answer:

    Your Python script, let's call it my_tests.py,

    from utils import test_wide_variable
    
    # gets used inside of test functions
    file_wide_variable = [1, 2, 3]
    
    def test_func():
        y = (file_wide_variable, test_wide_variable)
        # ...
    

    presumably gets called by the test harness with

    from inspect import getmembers, isfunction
    
    import my_tests
    for name, func in getmembers(my_tests, isfunction):
       func()
    

    or something similar (I haven't even tested this code).

    Your MATLAB script, let's call it my_tests.m,

    reset_test_params
    
    P.file_wide_variable = [1, 2, 3];
    P.test_wide_variable = load('test_wide_variables.m').test_wide_variable;
    
    function test_func(~)
        global P
        y = {P.file_wide_variable, P.test_wide_variable};
        # ...
    

    when called in a test harness,

    my_tests
    

    or equivalently

    run("my_tests")
    

    will cause P to be defined in the test harness's workspace (hopefully not overwriting any local variables!). test_func(), however, is lost. The script itself doesn't call it, so it remains unused and undefined. Being a local function, it can only be called from within my_tests.m.

    So, attempting to directly replicate Python's format for a test script is a non-starter, because the two languages work fundamentally differently in these respects.


    Note also that, when you call a script from within another script or function (in MATLAB), all of its variables will be defined in the caller's workspace. This means that the script can potentially overwrite any of the caller's variables. Using scripts to build a more complex piece of software is a really bad idea, it would be a nightmare trying to keep all of your script's variables separate. This is exactly what functions are for. Functions have a local scope, making composition a lot easier.

    I would not use scripts for anything other than (maybe) the program's entry point. Everything else should be a function. There is no reason not to use functions.


    This is how I would attempt to reproduce the Python format in MATLAB:

    test_wide_variable.m:

    function v = test_wide_variable
    v = [5, 6, 7];
    

    my_tests.m:

    function funcs = my_tests
       file_wide_variable = [1, 2, 3];
       data = test_wide_variable
    
       funcs = {@test_func};  % list all the test functions defined in the file
    
       function test_func
          y = {file_wide_variable, data};
          % ...
       end
    
    end
    

    Now this could be called from a test harness as follows:

    fail = 0
    total = 0
    for func = my_tests
       total = total + 1;
       try
          func()
       catch ME
          disp(ME)
          fail = fail + 1;
       end
    end
    disp(fail + "tests failed out of " + total " tests.")
    

    Note that the variables declared locally in the function my_tests() are available to the nested functions defined within this function (note that these functions are declared between the function and the end statements for my_tests(), and so they are nested functions, not local functions.

    The my_tests() function returns a cell array with handles to the nested functions. These can then be called by the calling test harness. When called, they still have access to the variables defined in my_tests as they were when the function handles were first obtained -- these are lambda captures.