Search code examples
androidgradleandroid-uiautomator

How to launch tests on APK from CI?

I have a CI server to build APK from GIT and it is working well.

I need to launch automatic tests (UI Automator) on builds produced by CI server. To achieve it I:

  1. setup CI server to build APK with tests by command gradlew assembleDebug. I also tried to build it with gradlew assembleDebug -Pandroid.injected.signing.store.file=/path/to/keystore.jks -Pandroid.injected.signing.store.password=password1 -Pandroid.injected.signing.key.alias=myapp -Pandroid.injected.signing.key.password=password2

  2. Trying to launch it with command adb.exe shell am instrument -w -r -e debug false -e package com.mysite.myapp com.mysite.myapp.test/com.mysite.myapp.runner.MyCustomTestsRunner

And no luck. I am getting an error:

java.lang.SecurityException: Permission Denial: starting instrumentation ComponentInfo{com.mysite.myapp.test/com.mysite.myapp.runner.MyCustomTestsRunner} from pid=4922, uid=4922 not allowed be
cause package com.mysite.myapp.test does not have a signature matching the target com.mysite.myapp
        at android.os.Parcel.createException(Parcel.java:2071)
        at android.os.Parcel.readException(Parcel.java:2039)
        at android.os.Parcel.readException(Parcel.java:1987)
        at android.app.IActivityManager$Stub$Proxy.startInstrumentation(IActivityManager.java:5441)
        at com.android.commands.am.Instrument.run(Instrument.java:512)
        at com.android.commands.am.Am.runInstrument(Am.java:196)
        at com.android.commands.am.Am.onRun(Am.java:80)
        at com.android.internal.os.BaseCommand.run(BaseCommand.java:56)
        at com.android.commands.am.Am.main(Am.java:50)
        at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
        at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:338)
INSTRUMENTATION_STATUS: Error=Permission Denial: starting instrumentation ComponentInfo{com.mysite.myapp.test/com.mysite.myapp.runner.MyCustomTestsRunner} from pid=4922, uid=4922 not allowed b
ecause package com.mysite.myapp.test does not have a signature matching the target com.mysite.myapp
INSTRUMENTATION_STATUS: id=ActivityManagerService
INSTRUMENTATION_STATUS_CODE: -1
Caused by: android.os.RemoteException: Remote stack trace:
        at com.android.server.am.ActivityManagerService.startInstrumentation(ActivityManagerService.java:15744)
        at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:2350)
        at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2741)
        at android.os.Binder.execTransactInternal(Binder.java:1021)
        at android.os.Binder.execTransact(Binder.java:994)

How does it works? I guess the APK already contains tests inside and there is no need to install some additional code... Or I have to build tests on CI too? How to launch tests on local computer using APK from CI?

Solution

  • You're probably having a signing issue with the debug variant of your app since the instrumented tests use the debug variant by default. Only way to tell is to check the build.gradle file of your app module and see what signingConfigs are applied.

    My suggestion is to try the following. Since eventually the CI is just running a command script, test this on your development machine and see if it succeeds:

    1. Open a command terminal and run the following commands which will build your application and the Android instrumented tests in debug variant and then install them both to a connected Android device:
      • gradlew clean
      • gradlew assembleDebug assembleDebugAndroidTest installDebug installDebugAndroidTest
    2. Launch the instrumented test cases:
      • adb.exe shell am instrument -w -r -e debug false -e package com.mysite.myapp com.mysite.myapp.test/com.mysite.myapp.runner.MyCustomTestsRunner

    If the exception java.lang.SecurityException is still appearing, then this should indicate more that something with signing of the APK is incorrect.