Search code examples
testing

Integration Test: Should test order matter?


I'm writing integration test for a standard crud app. Can get_user_api depend on creater_user_api? If each test case should be independent, then how can I maintain the state of database without creating and dropping tables between each tests?

Here is my creater_user_api

def create_user_api():
    send post request to register user x
    send get request to make sure user x is registered

send post request to register user x commits the changes to the database therefore I cannot perform rollback, the only way that I can think of to maintain the state of database is to drop all tables and create all tables between each tests, but this seems too inefficient.


Solution

  • Can get_user_api depend on create_user_api?

    Yes, this is an integration test. It tests the integration of units.

    Even in a unit test, a "unit" is not restricted to one method nor even one class but one behavior.

    Pragmatically, the simplest approach is to test each method with the other.

    I'd do it something like this...

    # Verify that get_user_api works when there are no users.
    # This is the zero/null case and can be completely isolated.
    def test_get_user_with_no_users:
      assert !get_user_api(username: "bob")
    
    # Test the full create, fetch, delete cycle.
    def test_create_get_delete_user:
      # Add another user to make user deleting doesn't delete all users.
      other = create_user_api(username: "other")
    
      # Create and fetch a user to verify creation worked.
      created_user = create_user_api(username: "bob")
      assert_eq get_user_api(username: "bob"), created_user
    
      # Delete a user and try to fetch it.
      delete_user_api(username: "bob")
      assert !get_user_api(username: "bob")
    
      # Fetch the other user to make sure only the one user got deleted.
      assert_eq get_user_api(username: "other"), other
    
    # Verify get_user_api will fail to find a user and won't, for example,
    # just pick some random user.
    def test_get_user_with_different_user:
      create_user_api(username: "bob")
      assert !get_user_api(username: "jill")
    

    If each test case should be independent, then how can I maintain the state of database without creating and dropping tables between each tests?

    Your test framework should run each test case in a transaction. Then when your code starts a transaction it will be a nested transaction. Its commits and rollbacks will only affect its own nested transaction. At the end of each test case, the test framework should rollback.

    Most databases don't support true nested transactions. They can be emulated using savepoint; your test or database framework can trap method calls to create a transaction and turn it into a savepoint. Here's a really rough sketch.

    def setup:
      db.begin()  # begins a transaction before every test case
    
    def teardown:
      db.rollback()  # rolls back the transaction after every test case
    
    def test_something:
      # Is already in a transaction.
      db.begin()    # begins a savepoint 
      db.commit()   # commits to the savepoint
    

    Most languages and frameworks already have ways to do this. You seem to be writing in Python. Django has TransactionTestCase which does this work for you. SQLAlchemy has nested transactions with which you can build a setup and teardown for your tests.