Search code examples
javaautomated-testsfunctional-testingregression-testingsmoke-testing

Automating smoke, functional and regression tests for Java web apps?

My understanding of a functional test is that its an integration test checking an entire, complete segment of application functionality; minus any stubs, mocks or other test doubles. For a web app this would be, perhaps, manually checking what happens when a user tries to enter bad credentials on a login form.

My understanding of a smoke test is that its a quick series of function tests that can be used to determine if the app has started up correctly and that its major subsystems are all online. For a web app, this might be to go to a health check URL and make sure that the app is up, and that it can reach all of its dependent subsystems (its database, its message broker, etc.).

My understanding of a regression test is that its a function test that is used in the future to determine if a particular function is no longer working as expected. As function tests are written for new features or rewritten for modifications to existing features, they are added to the regression test suite. In this sense, a regression test suite is the composition of all valid function tests for a particular web app.

Before I go any further, if anything I have said thus far is mislead or just not true, please begin by correcting me!

Assuming I'm more or less on track:

  1. Where do tools like Selenium WebDriver or BrowserStack get categorized here? Are they for smoke or regression tests?
  2. What are the generally-accepted methods/tools for conducting these sorts of tests?
  3. What are the generally-accepted methods from launching/running these tests from an automated build (such as Jenkins or Bamboo)?
Solution

  • Generally, Considering the automation of web-app, we have the following stages

    • Automate the Sanity/Smoke Tests

    To do this, we need to make sure that the automation criteria for the sanity tests can be achieved using the Test Automation Tool. There is no hard rule, to use a single tool to achieve sanity automation across all the domains for a web-app. Once the sanity is stabilized, we go ahead with the next.

    • Automate the Release / Regression Suite

    Automation of Release suite or Regression suite, must be cost-effective in manner, and the test automation framework design plays a important role over here since the tests need a long term maintenance

    For your questions my response is as follows

    1. It depends on the project strategy.

    2. What I suggest is to opt for automated sanity test, in a way, so that automation can be done quickly. If we start full fledged Test Automation Framework Development here itself, we may be taking more time for sanity automation

      And for Release Automation, carefully plan the Framework Architecture, considering all the possibilites and requirements of Framework

    3. We can integrate the Automated Tests with any CI Server like Jenkins or Bamboo or cruise and it is all dependent on the automation strategy

    Considering my experience, this is what I do if I were you

    • First, identify the tool acceptible for automation over that domain
    • Automate Sanity, with a basic framework, which can run tests and send out email with the report to stakeholders
    • Establish a CI server, and create a job, so that it runs on a daily basis and all the stakeholders are recieving results
    • Start Automation Approach for Release Suite which includes the Test Automation Framework Development