Signature mismatch between apk & espresso apk when running tests on Browserstack automate

Question

I've written a pipeline on Azure that builds our apk, builds the espresso test apk, signs them both with the Keystore that's located in secure storage, then uploads them to BrowserStack automate & executes the test. The issue occurs at the Builds dashboard on Browserstack where it says :

There is signature mismatch between application APK and test-suite APK. Package com.example.appName.test does not have a signature matching the target package com.example.appName. Please fix this, upload your app & test-suite APK and execute the test again.

Here's my.yml file:

pool:
  vmImage: ubuntu-latest

steps:
# download .jks file from secure files for signing
- task: DownloadSecureFile@1
  name: keystore
  displayName: 'Download keystore file'
  inputs:
    secureFile: 'appName.jks'

- script: |
    echo Installing $(keystore.secureFilePath) to the trusted keystore directory...
    sudo chown root:root $(keystore.secureFilePath)
    sudo chmod a+r $(keystore.secureFilePath)
    sudo ln -s -t /etc/ssl/certs/ $(keystore.secureFilePath)
  displayName: "Installing keystore file"

# build apk
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    publishJUnitResults: false
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'assembleGoogleDebug'
  displayName: "Building default APK"

# Build test apk
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    publishJUnitResults: false
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'assembleGoogleDebugAndroidTest'
  displayName: "Building test APK"

# sign the apks with the same signature
- task: AndroidSigning@3
  inputs:
    apkFiles: '**/*.apk'
    apksignerKeystoreFile: 'appName.jks'
    apksignerKeystorePassword: '***'
    apksignerKeystoreAlias: 'appName'
    apksignerKeyPassword: '***'
    zipalign: false
  displayName: "Signing APKs"

- task: BrowserStackConfig@0
  inputs:
    BrowserStackServiceEndPoint: 'BrowserStack connection'
  displayName: "Establishing BrowserStack connection..."

# Upload apk to BrowserStack
- task: BrowserStackAppUploader@0
  inputs:
    appPath: '/home/vsts/work/1/s/presentation/build/outputs/apk/google/debug/presentation-google-debug.apk'
    appCustomId: 'sport24'
  displayName: 'Uploading default APK to BrowserStack'

- script: |
   curl -u "name:key" \
   -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/test-suite" \
   -F "file=@ /home/vsts/work/1/s/presentation/build/outputs/apk/androidTest/google/debug/presentation-google-debug-androidTest.apk"
   -F "custom_id=appNameTest"
   echo "triggering test" 
   
   curl -u "name:key" \
   -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
   -d '{"app": "url", "testSuite": "bs://url", "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
   -H "Content-Type: application/json" 
  displayName: 'uploading test suite and triggering test'

# post test results
- task: BrowserStackResults@0

I'm signing both APKs with the same Keystore & the test class is located under com.example.appName/androidTest/ so I can't figure out what's causing the mismatch. Any ideas?

Answer 1

This error seems to occur for a variety of reasons.

Looking at your .yml file I can suggest the following 2 corrections:

1. You are missing a backslash \ right before the -F parameter where you specify the custom id of your test suite, this is required otherwise the custom id for the test suite will not be set, So replace this line:

-F "file=@ /home/vsts/work/1/s/presentation/build/outputs/apk/androidTest/google/debug/presentation-google-debug-androidTest.apk"

with this line (just add the \ at the end):

-F "file=@ /home/vsts/work/1/s/presentation/build/outputs/apk/androidTest/google/debug/presentation-google-debug-androidTest.apk" \

2. In your final curl command where you trigger the execution of the tests, you should replace the URLs with the custom ids you set for the app and the test suit.

So edit this line:

-d '{"app": "url", "testSuite": "bs://url", "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \

To this:

-d '{"app": "INSERT_CUSTOM_APP_ID_HERE", "testSuite": "INSERT_CUSTOM_TEST_ID_HERE", "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \

What the above change basically does is that you now specify the app and test suite for test execution using the ids you previously set in contrast to using the URL's that differ with each execution probably causing the error in the first place.