Search code examples
pythonunit-testingfunctional-testing

Is there an alternative result for Python unit tests, other than a Pass or Fail?


I'm writing unit tests that have a database dependency (so technically they're functional tests). Often these tests not only rely on the database to be live and functional, but they can also rely on certain data to be available.

For example, in one test I might query the database to retrieve sample data that I am going to use to test the update or delete functionality. If data doesn't already exist, then this isn't exactly a failure in this context. I'm only concerned about the pass/fail status of the update or delete, and in this situation we didn't even get far enough to test it. So I don't want to give a false positive or false negative.

Is there an elegant way to have the unit test return a 3rd possible result? Such as a warning?


Solution

  • In general I think the advice by Paul Becotte is best for most cases:

    This is a failure though- your tests failed to set up the system in the way that your test required. Saying "the data wasn't there, so it is okay for this test to fail" is saying that you don't really care about whether the functionality you are testing works. Make your test reliably insert the data immediately before retrieving it, which will give you better insight into the state of your system.

    However, in my particular case, I am writing a functional test that relies on data generated and manipulated from several processes. Generating it quickly at the beginning of the test just isn't practical (at least yet).

    What I ultimately found to work as I need it to use skipTest as mentioned here:

    Skip unittest if some-condition in SetUpClass fails