How to use iOS with Robot Framework in Plone

Question

I'm writing Robot Framework tests for a Plone project.

Here is the code.

I want tests to be executed on Sauce Labs, but with two different OS's:

• iOS
• Android

I'm starting by replacing the current default tests (Firefox / WinXP) with iOS, using the following parameters in my .travis.yml's matrix:

  - TARGET=test ROBOT_DESIRED_CAPABILITIES="platform:OS X 10.8,browserName:iPad,version:6,tunnel-identifier:$TRAVIS_JOB_ID"

Tests are now failing with the following issues:

 Could not proxy http://localhost:3001/wd/hub/session, exception: java.net.ConnectException: Connection refused

I have no idea why. I have read all the stuff in example.product and I didn't find any differences.

Here are my builds at Travis CI, which show the errors.

And you can find my Sauce Labs results here.

Answer 1

Currently, you have to run your test ZServer (by settings ZSERVER_PORT environment variable) at some port supported by Sauce Labs: https://saucelabs.com/docs/connect#localhost

• I have a working example at: https://github.com/datakurre/example.product

• I filed a request for SauceLabs to add support for port 55001 used by Plone, and the port will be included in the next SauceConnect-release.

Update: There was an another issue. p.a.robotframework defines Selenium version for Sauce Labs, currently 2.31.0 (UPDATE: next release will not set the default value). I asked this from Sauce Labs and it seems that selenium version should not be set for mobile tests at all.

• I have now a working p.a.robotframework example at: https://github.com/datakurre/example.product/tree/p.a.robotframework

• To reset Sauce Labs Selenium version set by p.a.robotframework, you could set an environment variable ROBOT_SELENIUM_VERSION=" " for mobile Travis builds as in https://github.com/datakurre/example.product/commit/4deafc843b89a66da8cf6a85955a2b936ada79b4

Technically ROBOT_SELENIUM_VERSION="" should be enough for p.a.robotframework to use the default Selenium version of Sauce Labs, but when I tested that in .travis.yml, Travis didn't set that environment variable at all and it didn't override p.a.robotframework's default. Anyway, the next p.a.robotframework-release (>= 0.6.2) will set no selenium-version for Sauce Labs by default.

Background:

• Sauce Labs documentation related to Selenium version: https://saucelabs.com/docs/additional-config#selenium-version

• p.a.robotframework code for defining SELENIUM_VERSION-variable for Robot Framework and forcing selenium-version Selenium desired capabilities string for Sauce Labs: https://github.com/plone/plone.app.robotframework/blob/master/src/plone/app/robotframework/saucelabs.robot#L25

• Finally robotsuite, which is used used to wrap Robot Framework tests for zope.testrunner, passes all ROBOT_-prefixed environment variables for Robot Framework test runner without the prefix, effectively overriding the default set in p.a.robotframework.