Search code examples
androidjenkinsdockerubuntu-16.04calabash-android

Calabash android.util.AndroidException: INSTRUMENTATION_FAILED


The situation is as follows. I created a headless Ubuntu 16.04 Docker container with Jenkins 2.32.2, at the moment openjdk 1.8.0_121 as well as JDK 7u80 (the JDK 7 is used for the build), android SDK 16, 18, 23, 24 and 25.

Somehow this setup is not working that well with my calabash tests since i always get a error message, wenn i try to run the tests.

I installed ruby 2.3.1p112 and have the following gems:

awesome_print (1.7.0)
bigdecimal (1.2.8)
builder (3.2.3)
calabash-android (0.9.0)
cucumber (2.4.0)
cucumber-core (1.5.0)
cucumber-wire (0.0.1)
did_you_mean (1.0.0)
diff-lcs (1.3)
escape (0.0.4)
gherkin (4.0.0)
httpclient (2.8.3)
io-console (0.4.5)
json (1.8.3)
luffa (2.0.0)
minitest (5.8.4)
multi_json (1.12.1)
multi_test (0.1.2)
net-telnet (0.1.1)
power_assert (0.2.7)
psych (2.0.17)
rake (10.5.0)
rdoc (4.2.1)
retriable (2.0.2)
rubyzip (1.2.1)
slowhandcuke (0.0.3)
test-unit (3.1.7)
thor (0.19.4)

In the jenkins i have a job that creates and starts a Emulator with the folowing commands:

08:35:13 [android] Creating Android AVD: /root/.android/avd/hudson_de-DE_160_WVGA_android-16_armeabi-v7a_jenkinsandroidemulator.avd
08:35:13 [android] /opt/android-sdk-linux/tools/android create avd -f -a -c 200M -s WVGA800 -n hudson_de-DE_160_WVGA_android-16_armeabi-v7a_jenkinsandroidemulator -t android-16 --abi armeabi-v7a
08:35:14 $ /opt/android-sdk-linux/platform-tools/adb start-server
08:35:17 * daemon not running. starting it now on port 5848 *
08:35:17 * daemon started successfully *
08:35:17 $ /opt/android-sdk-linux/platform-tools/adb start-server
08:35:17 [android] Starting Android emulator
08:35:17 $ /opt/android-sdk-linux/tools/emulator -ports 5754,5755 -report-console tcp:5849,max=60 -prop persist.sys.language=de -prop persist.sys.country=DE -avd hudson_de-DE_160_WVGA_android-16_armeabi-v7a_jenkinsandroidemulator -no-snapshot-load -no-snapshot-save -wipe-data -no-window -force-32bit
08:35:18 emulator: warning: opening audio output failed
08:35:18 
08:35:23 [android] Emulator reported that the console is available on port 5,754
08:35:23 [android] Waiting for emulator to finish booting...
08:35:23 $ /opt/android-sdk-linux/platform-tools/adb -s emulator-5754 wait-for-device shell getprop init.svc.bootanim
08:35:23 emulator: UpdateChecker: skipped version check
08:35:42 [android] Emulator reported that the startup process is 'running'
08:35:55 $ /opt/android-sdk-linux/platform-tools/adb -s emulator-5754 wait-for-device shell getprop init.svc.bootanim
08:36:36 [android] Emulator reported that the startup process is 'running'
08:36:50 $ /opt/android-sdk-linux/platform-tools/adb -s emulator-5754 wait-for-device shell getprop init.svc.bootanim
08:36:50 [android] Emulator reported that the startup process is 'stopped'
08:36:50 $ /opt/android-sdk-linux/platform-tools/adb -s emulator-5754 logcat -v time
08:36:50 [android] Emulator is ready for use (took 92 seconds)

afterwards the job builds my android project using gradle. After the apk is build the calabash tests should run, however i get the following output:

08:41:30 + calabash-android run /var/jenkins_home/workspace/Appname/app/build/outputs/apk/app-debug.apk --verbose --format json --out result.json
08:41:31 No test server found for this combination of app and calabash version. Recreating test server.
08:41:31 [35mDEBUG: Setting Android SDK location to $ANDROID_HOME[0m
08:41:31 [35mDEBUG: Android SDK location set to '/opt/android-sdk-linux'[0m
08:41:31 [35mDEBUG: Set aapt path to '/opt/android-sdk-linux/build-tools/25.0.2/aapt'[0m
08:41:31 [35mDEBUG: Set zipalign path to '/opt/android-sdk-linux/build-tools/25.0.2/zipalign'[0m
08:41:31 [35mDEBUG: Set adb path to '/opt/android-sdk-linux/platform-tools/adb'[0m
08:41:31 [35mDEBUG: Set android jar path to '/opt/android-sdk-linux/platforms/android-25/android.jar'[0m
08:41:31 [35mDEBUG: Setting Java SDK location to $JAVA_HOME[0m
08:41:31 [35mDEBUG: Java SDK location set to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7'[0m
08:41:31 [35mDEBUG: Found java on PATH[0m
08:41:31 [35mDEBUG: Set java path to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/java'[0m
08:41:31 [35mDEBUG: Found keytool on PATH[0m
08:41:31 [35mDEBUG: Set keytool path to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/keytool'[0m
08:41:31 [35mDEBUG: Found jarsigner on PATH[0m
08:41:31 [35mDEBUG: Set jarsigner path to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/jarsigner'[0m
08:41:31 2017-02-22 08:41:31 - Signature files:
08:41:31 2017-02-22 08:41:31 - /tmp/d20170222-4136-1g8arof/META-INF/CERT.RSA
08:41:31 2017-02-22 08:41:31 - "/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/keytool" -v -printcert -J"-Dfile.encoding=utf-8" -file "/tmp/d20170222-4136-1g8arof/META-INF/CERT.RSA"
08:41:31 2017-02-22 08:41:31 - SHA1 fingerprint for signing cert (/var/jenkins_home/workspace/Appname/app/build/outputs/apk/app-debug.apk): DC:DD:67:19:78:DE:43:A1:7E:E8:26:DE:DA:43:78:0C
08:41:31 2017-02-22 08:41:31 - /var/jenkins_home/workspace/Appname/app/build/outputs/apk/app-debug.apk was signed with a certificate with fingerprint DC:DD:67:19:78:DE:43:A1:7E:E8:26:DE:DA:43:78:0C
08:41:31 2017-02-22 08:41:31 - Reading keystore data from keystore file '/root/.android/debug.keystore'
08:41:31 2017-02-22 08:41:31 - "/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/keytool" -list -v -alias androiddebugkey -keystore /root/.android/debug.keystore -storepass android "-J"-Dfile.encoding=utf-8"" "-J"-Duser.language=en-US""
08:41:31 2017-02-22 08:41:31 - Key store data:
08:41:31 2017-02-22 08:41:31 - Alias name: androiddebugkey
08:41:31 Creation date: Feb 21, 2017
08:41:31 Entry type: PrivateKeyEntry
08:41:31 Certificate chain length: 1
08:41:31 Certificate[1]:
08:41:31 Owner: CN=Android Debug, O=Android, C=US
08:41:31 Issuer: CN=Android Debug, O=Android, C=US
08:41:31 Serial number: 5c473d
08:41:31 Valid from: Tue Feb 21 08:48:21 UTC 2017 until: Thu Feb 14 08:48:21 UTC 2047
08:41:31 Certificate fingerprints:
08:41:31     MD5:  E4:12:04:8F:43:6F:90:8A:CA:48:41:B8:48:E3:18:CD
08:41:31     SHA1: CE:AA:E1:80:DC:DD:67:19:78:DE:43:A1:7E:E8:26:DE:DA:43:78:0C
08:41:31     SHA256: 50:8E:75:7F:1D:A0:2B:01:49:B2:CF:EB:A1:58:16:C5:2C:22:15:44:67:C9:B9:28:8B:C0:FB:F5:A4:6F:2F:22
08:41:31     Signature algorithm name: SHA256withRSA
08:41:31     Version: 3
08:41:31 
08:41:31 Extensions: 
08:41:31 
08:41:31 #1: ObjectId: 2.5.29.14 Criticality=false
08:41:31 SubjectKeyIdentifier [
08:41:31 KeyIdentifier [
08:41:31 0000: FF 61 B4 C8 D9 85 A2 88   E5 D7 E6 39 58 C1 B5 91  .a.........9X...
08:41:31 0010: 7A 8B 44 88                                        z.D.
08:41:31 ]
08:41:31 ]
08:41:31 
08:41:31 2017-02-22 08:41:31 - Fingerprint: DC:DD:67:19:78:DE:43:A1:7E:E8:26:DE:DA:43:78:0C
08:41:31 2017-02-22 08:41:31 - Signature algorithm name: SHA256withRSA
08:41:31 2017-02-22 08:41:31 - Unlocked keystore at /root/.android/debug.keystore - fingerprint: DC:DD:67:19:78:DE:43:A1:7E:E8:26:DE:DA:43:78:0C
08:41:31 2017-02-22 08:41:31 - Trying to read keystore from: /var/jenkins_home/workspace/Appname/calabash_tests/debug.keystore - no such file
08:41:31 2017-02-22 08:41:31 - Trying to read keystore from: /root/.local/share/Xamarin/Mono\ for\ Android/debug.keystore - no such file
08:41:31 2017-02-22 08:41:31 - Trying to read keystore from: /root/AppData/Local/Xamarin/Mono for Android/debug.keystore - no such file
08:41:31 2017-02-22 08:41:31 - Signing using the signature algorithm: 'SHA1withRSA'
08:41:31 2017-02-22 08:41:31 - Signing using the digest algorithm: 'SHA1'
08:41:31 2017-02-22 08:41:31 - "/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/jarsigner" -sigfile CERT -sigalg SHA1withRSA -digestalg SHA1 -signedjar test_servers/3ddf16f67946400b78f7cc84e43884f0_0.9.0.apk -storepass android -keystore /root/.android/debug.keystore /tmp/d20170222-4136-1cudbte/TestServer.apk androiddebugkey
08:41:32 Done signing the test server. Moved it to test_servers/3ddf16f67946400b78f7cc84e43884f0_0.9.0.apk
08:41:32 2017-02-22 08:41:32 - "/usr/bin/ruby2.3" -S cucumber --verbose --format json --out result.json APP_PATH="/var/jenkins_home/workspace/Appname/app/build/outputs/apk/app-debug.apk" TEST_APP_PATH="test_servers/3ddf16f67946400b78f7cc84e43884f0_0.9.0.apk"
08:41:33 Code:
08:41:33   * features/support/env.rb
08:41:33 [35mDEBUG: Setting Android SDK location to $ANDROID_HOME[0m
08:41:33 [35mDEBUG: Android SDK location set to '/opt/android-sdk-linux'[0m
08:41:33 [35mDEBUG: Set aapt path to '/opt/android-sdk-linux/build-tools/25.0.2/aapt'[0m
08:41:33 [35mDEBUG: Set zipalign path to '/opt/android-sdk-linux/build-tools/25.0.2/zipalign'[0m
08:41:33 [35mDEBUG: Set adb path to '/opt/android-sdk-linux/platform-tools/adb'[0m
08:41:33 [35mDEBUG: Set android jar path to '/opt/android-sdk-linux/platforms/android-25/android.jar'[0m
08:41:33 [35mDEBUG: Setting Java SDK location to $JAVA_HOME[0m
08:41:33 [35mDEBUG: Java SDK location set to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7'[0m
08:41:33 [35mDEBUG: Found java on PATH[0m
08:41:33 [35mDEBUG: Set java path to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/java'[0m
08:41:33 [35mDEBUG: Found keytool on PATH[0m
08:41:33 [35mDEBUG: Set keytool path to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/keytool'[0m
08:41:33 [35mDEBUG: Found jarsigner on PATH[0m
08:41:33 [35mDEBUG: Set jarsigner path to '/var/jenkins_home/tools/hudson.model.JDK/JDK_7/bin/jarsigner'[0m
08:41:33   * features/support/app_installation_hooks.rb
08:41:33   * features/support/app_life_cycle_hooks.rb
08:41:33   * features/support/hooks.rb
08:41:33   * features/step_definitions/Custom_Step_Definition.rb
08:41:33   * features/step_definitions/calabash_steps.rb
08:41:33 
08:41:33 Features:
08:41:33   * features/registrierung_karteikarten_check.feature
08:41:33 
08:41:33 2017-02-22 08:41:33 - connected_devices: ["emulator-5754"]
08:41:33 2017-02-22 08:41:33 - Persistently allocated port 34784 to emulator-5754
08:41:33 2017-02-22 08:41:33 - "/opt/android-sdk-linux/platform-tools/adb" -s emulator-5754 forward tcp:34784 tcp:7102
08:41:33 2017-02-22 08:41:33 - 
08:41:33 2017-02-22 08:41:33 - Starting test server using:
08:41:33 2017-02-22 08:41:33 - "/opt/android-sdk-linux/platform-tools/adb" -s emulator-5754 shell am instrument -e main_activity null -e test_server_port 7102 -e class sh.calaba.instrumentationbackend.InstrumentationBackend com.appname.appname2.test/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner
08:41:34 android.util.AndroidException: INSTRUMENTATION_FAILED: com.appname.appname2.test/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner
08:41:34    at com.android.commands.am.Am.runInstrument(Am.java:616)
08:41:34    at com.android.commands.am.Am.run(Am.java:118)
08:41:34    at com.android.commands.am.Am.main(Am.java:81)
08:41:34    at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
08:41:34    at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:235)
08:41:34    at dalvik.system.NativeStart.main(Native Method)
08:41:41 2017-02-22 08:41:34 - It looks like your app is no longer running. 
08:41:41 It could be because of a crash or because your test script shut it down.
08:41:41 2017-02-22 08:41:34 - It looks like your app is no longer running. 
08:41:41 It could be because of a crash or because your test script shut it down.

Here the logcat output, while calabash is working:

D/AndroidRuntime(  629): >>>>>> AndroidRuntime START com.android.internal.os.RuntimeInit <<<<<<
D/AndroidRuntime(  629): CheckJNI is ON
D/dalvikvm(  629): Trying to load lib libjavacore.so 0x0
D/dalvikvm(  629): Added shared lib libjavacore.so 0x0
D/dalvikvm(  629): Trying to load lib libnativehelper.so 0x0
D/dalvikvm(  629): Added shared lib libnativehelper.so 0x0
D/AndroidRuntime(  629): Calling main entry com.android.commands.am.Am
W/ActivityManager(  148): Unable to find instrumentation info for: ComponentInfo{com.appname.appname2.test/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner}
I/AndroidRuntime(  629): VM exiting with result code 1.

So i have already searched for a solution for several hours but nothing seems to work. Here some more infos when i try to reproduce the messages manually.

Create avd: /opt/android-sdk-linux/tools/android create avd -f -a -c 200M -s WVGA800 -n hudson_de-DE_160_WVGA_android-16_armeabi-v7a_jenkinsandroidemulator -t android-16 --abi armeabi-v7a

Start server: /opt/android-sdk-linux/platform-tools/adb start-server

Start Emulator: /opt/android-sdk-linux/tools/emulator -ports 5754,5755 -prop persist.sys.language=de -prop persist.sys.country=DE -avd hudson_de-DE_160_WVGA_android-16_armeabi-v7a_jenkinsandroidemulator -no-snapshot-load -no-snapshot-save -wipe-data -no-window -force-32bit

Difference to Jenkins command is that the emulator ist startet without "-report-console tcp:5849,max=60" After that i run: calabash-android run app-debug.apk --verbose --format json --out result.json

I got the same output. I tried calabash-android resign app-debug.apk. Also that with calabash-android built app-debug.apk i still get the same output.

I opened the console via calabash-android console app-debug.apk and did reinstall_apps and afterwards run start_test_server_in_background With that i had a new problem the emulator was killed and i got the following answer from the test server:

Errno::ECONNREFUSED: Connection refused - Connection refused - connect(2) for "127.0.0.1" port 34785 (127.0.0.1:34785)
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:623:in `rescue in create_socket'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:607:in `create_socket'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:755:in `block in connect'
        from /usr/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
        from /usr/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:748:in `connect'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:511:in `query'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:177:in `query'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient.rb:1242:in `do_get_block'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient.rb:1138:in `rescue in protect_keep_alive_disconnected'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient.rb:1132:in `protect_keep_alive_disconnected'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
        from /var/lib/gems/2.3.0/gems/httpclient-2.8.3/lib/httpclient.rb:765:in `post'
        from /var/lib/gems/2.3.0/gems/calabash-android-0.9.0/lib/calabash-android/operations.rb:531:in `make_http_request'
        from /var/lib/gems/2.3.0/gems/calabash-android-0.9.0/lib/calabash-android/operations.rb:468:in `http'
        from /var/lib/gems/2.3.0/gems/calabash-android-0.9.0/lib/calabash-android/operations.rb:800:in `start_application'
        from /var/lib/gems/2.3.0/gems/calabash-android-0.9.0/lib/calabash-android/operations.rb:790:in `start_test_server_in_background'
        from /var/lib/gems/2.3.0/gems/calabash-android-0.9.0/lib/calabash-android/operations.rb:152:in `start_test_server_in_background'
        from (irb):1
        from /usr/bin/irb:11:in `<main>'

In the console that run the emulator the output is the following:

Segmentation fault (core dumped)

I also checked if there is a problem with the Instrumentations and befor i do reinstall_apps in the calabash console the available Instruementations are this:

instrumentation:com.android.emulator.connectivity.test/android.test.InstrumentationTestRunner (target=com.android.emulator.connectivity.test)
instrumentation:com.android.emulator.gps.test/android.test.InstrumentationTestRunner (target=com.android.emulator.gps.test)
instrumentation:com.android.smoketest.tests/com.android.smoketest.SmokeTestRunner (target=com.android.smoketest)
instrumentation:com.android.smoketest.tests/android.test.InstrumentationTestRunner (target=com.android.smoketest)
instrumentation:com.example.android.apis/.app.LocalSampleInstrumentation (target=com.example.android.apis)

After the reinstall_apps it is as follows:

instrumentation:com.android.emulator.connectivity.test/android.test.InstrumentationTestRunner (target=com.android.emulator.connectivity.test)
instrumentation:com.android.emulator.gps.test/android.test.InstrumentationTestRunner (target=com.android.emulator.gps.test)
instrumentation:com.android.smoketest.tests/com.android.smoketest.SmokeTestRunner (target=com.android.smoketest)
instrumentation:com.android.smoketest.tests/android.test.InstrumentationTestRunner (target=com.android.smoketest)
instrumentation:com.appname.appname2.test/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner (target=com.appname.appname2)
instrumentation:com.appname.appname2.test/sh.calaba.instrumentationbackend.ClearAppData (target=com.appname.appname2)
instrumentation:com.appname.appname2.test/sh.calaba.instrumentationbackend.ClearAppData2 (target=com.appname.appname2)
instrumentation:com.appname.appname2.test/sh.calaba.instrumentationbackend.ClearPreferences (target=com.appname.appname2)
instrumentation:com.appname.appname2.test/sh.calaba.instrumentationbackend.SetPreferences (target=com.appname.appname2)
instrumentation:com.appname.appname2.test/sh.calaba.instrumentationbackend.GetPreferences (target=com.appname.appname2)
instrumentation:com.example.android.apis/.app.LocalSampleInstrumentation (target=com.example.android.apis)

Solution

  • Well it would appear all the logoutput was missing a importent message. After some more testing i got this message:

    `Cucumber::Ast` no longer exists. These classes have moved into the `Cucumber::Core::Ast` namespace, but may not have the same API. (RuntimeError)
    

    Now i know what my problem was. It was the wrong cucumber version which made the trouble. I now have cucumber 1.3.20 and i can at least somewhat proceed i have a new error but for now lets see what i can find to that elsewhere.