Can Appium run multiple different tests concurrently across multiple devices?

Question

I am new to Appium/Selenium parallel testing and I was wondering if one could run different tests concurrently across multiple devices? My team needs to reduce the total runtime of our UI tests and are not concerned with different OS versions affecting the behaviour of the application for these specific tests. I have been reading through many posts and trying to search for answers but all I can seem to find on the internet are articles, tutorials and forums on how to run the same test in parallel on multiple devices.

Can I run different tests concurrently on multiple devices without kicking off different tests manually, or is that a limitation of Appium? Ideally this would be implemented using an open source solution.

(Right now we are trying to use a JUnit approach for testing due to specific limitations of other tools. All tests are being written in Java.)

Thanks for your time.

Answer 1

Depending on your setup, you can accomplish this. However, a lot of your build automation and device management will need to be set up by you or your team custom, so you will not be able to use an out-of-the-box solution to do this.

I've accomplished the same with both Selenium and Appium -- you will need a test framework that allows for test execution with parameters, and your devices will need to be connected to separate USB hubs that each have their own virtual server attached.

Using NUnit, here's my approach:

• Generate .txt files for each different set of tests I want to run -- test_list_1.txt, test_list_2.txt, etc. Each list contains a different group of test cases to run.
• Write a build script to clean & build your project from scratch -- for C#, I use Cake.
• Set up a job in Jenkins that executes your build script and calls NUnit's console runner, which takes a test_list as a parameter. This initiates a test execution against a list of test cases

You should be able to build your Jenkins job against any test list you want, so you now have the ability to run your automation against different tests, as mentioned in your problem description.

Connect your virtual machines (which connect to your Appium devices) to Jenkins and add them as executors on your job. Now you have multiple machines to run your job against.

With this set up, you can run as many jobs as you have machines -- 4 VM's means 4 jobs, which means you can run 4 different sets of test cases concurrently.

Setting this up on my end was completely custom -- I used certain tools to accomplish individual steps, but it worked for our needs and we did accomplish concurrent execution against different sets of test cases.