Search code examples
emacsert

Can Emacs ERT tests order be changed?


I've noticed that ERT tests (ert-run-tests-batch-and-exit) in Emacs are executed by name (I mean order).
Can I somehow execute tests in the order I define them?

For example:

(ert-deftest test-2 ...
(ert-deftest test-1 ...

will execute first "test-2", then "test-1".

PS: I know that tests should not rely on the order but still want it.


Solution

  • As far as I know you can't tell ERT to execute tests in the order in which they are defined. You can, however, specify the order in which you would like your tests to be executed manually, by passing an appropriate SELECTOR argument to ERT's functions for running tests. You can try this out interactively by doing:

    M-x ert RET (member test-2 test-1) RET

    You might want to make test-2 and test-1 fail temporarily so you can verify that ERT runs them in the order you specified.

    Now, based on the fact that you mentioned ert-run-tests-batch-and-exit in your question I am assuming you want to run tests from the command line. The following command will do what you want (where <test-file> is the name of the file containing your tests):

    emacs -batch -l ert -l <test-file> --eval "(ert-run-tests-batch-and-exit '(member test-2 test-1))"
    

    Bonus

    If you want to avoid having to specify the order in which you want your tests to run every time you execute this command, you can add the following to your <test-file>:

    (defvar test-order '(member test-2 test-1))
    

    With this in place you can run your tests from the command line via:

    emacs -batch -l ert -l <test-file> --eval "(ert-run-tests-batch-and-exit test-order)"