This is for a largish 3-4 year running Android project, running on Gradle 5.4.1. Integration testing is with Mockito, espresso and dagger.
I have run into an issue where we are adding a Pendo library to the project, the dependency was added to Gradle as standard. Everything runs fine, until we try to run integration tests (~2000), these are run in shards with Spoon.
Around half way through the integration tests, on random tests each time, we run into a native crash killing the test run, due to LinearAlloc exceeding capacity. Running these tests in isolation, or in their classes locally they pass with no issues and have been stable for a long time.
I brought the whole app back to the known good build, added the Pendo dependency only and this results in the same problem, however I don't believe this is due to Pendo, as I tested by coming back to a known good build (tested on again at this point for sanity) and adding a random new dependency, this resulted in the same problem.
From what I can find this may be something to do with the method limits around Android. I should mention we are using multidex to break the app down. Proguard and minify are also being used.
Part of the issue here is that I'm really not sure what to look at to figure out what's going on to cause this overflow. Following the logs for the test runs, nothing appears to be amiss, bar a fair bit of garbage collection (which I'm guessing means a leak somewhere). I'm unsure if this issue is down to some underlying leak, and the new libraries are pushing something just over the edge, or if there's some dependency limit in android that I'm unaware of, or some other way to break the files down so we aren't causing LinearAlloc to fill up.
From reading, I know the limits of LinearAlloc were upped around Android 5, we are having problems on devices both above (Android 10) and below this (Android 4) and I don't really see much chat around this since 2017, so I feel like I'm missing something obvious, or something is misconfigured in the project given it was setup before then.
Any help would be really appreciated. I've dumped a cut down version of the gradle file below
android {
compileSdkVersion 28
buildToolsVersion "28.0.3"
sourceSets {
main.java.srcDirs += "src/main/kotlin"
main.jniLibs.srcDirs = ["src/main/jni"]
androidTest.java.srcDirs += ["src/androidTest/kotlin", "src/testUtilities/kotlin"]
debug.java.srcDirs += "src/debug/kotlin"
test.java.srcDirs += ["src/test/kotlin", "src/testUtilities/kotlin"]
}
defaultConfig {
minSdkVersion 17
targetSdkVersion 25 // Required to be sdk 25
versionCode 94
versionName "3.4.0"
versionNameSuffix System.getenv("ANDROID_VERSION_NAME_SUFFIX") ?: ""
multiDexEnabled true
multiDexKeepProguard file("multidex-keep-rules.pro")
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
testBuildType "debug"
renderscriptTargetApi 17
renderscriptSupportModeEnabled false
vectorDrawables.useSupportLibrary = true
}
packagingOptions {
exclude "META-INF/ASL2.0"
exclude "META-INF/LICENSE"
exclude "META-INF/NOTICE"
exclude "META-INF/NOTICE.txt"
exclude "META-INF/LICENSE.txt"
exclude "META-INF/MANIFEST.MF"
exclude "META-INF/main.kotlin_module"
exclude "META-INF/proguard/androidx-annotations.pro"
exclude "META-INF/atomicfu.kotlin_module"
}
buildTypes {
debug {
minifyEnabled true
shrinkResources true
proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
testCoverageEnabled = false
applicationIdSuffix = '.dev'
versionNameSuffix "-debug"
manifestPlaceholders = [isDebug: true]
}
release {
minifyEnabled true
shrinkResources true
proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
signingConfig signingConfigs.release
manifestPlaceholders = [isDebug: false]
}
}
dataBinding {
enabled = true
}
testOptions {
animationsDisabled = true
}
lintOptions {
checkDependencies = false
checkGeneratedSources = false
ignoreTestSources = true
disable "ExpiredTargetSdkVersion"
}
externalNativeBuild {
cmake {
path "CMakeLists.txt"
}
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
}
composer {
variants "debug"
shard true
withOrchestrator false
verboseOutput false
keepOutput true
if (project.hasProperty("composerClassTarget")) {
instrumentationArgument("class", project.getProperty("composerClassTarget"))
}
}
configurations {
ktlint
}
task ktlint(type: JavaExec) {
main = "com.pinterest.ktlint.Main"
classpath = configurations.ktlint
args "src/**/*.kt"
}
check.dependsOn ktlint
detekt {
input = files("$project.projectDir/src")
config = files("$project.projectDir/detekt.yml")
filters = ".*/testUtilities/.*, .*build.*, .*/tmp/.*, .*/resources/.*, .*/res/.*"
parallel = true
}
dependencies {
ktlint "com.pinterest:ktlint:0.36.0"
kapt "com.google.dagger:dagger-compiler:$daggerVersion"
kapt "com.github.bumptech.glide:compiler:$glideVersion"
kaptAndroidTest "com.google.dagger:dagger-compiler:$daggerVersion"
implementation fileTree(include: ['**/*.jar'], dir: 'libs')
implementation "androidx.multidex:multidex:2.0.1"
implementation "androidx.legacy:legacy-support-v4:1.0.0"
implementation "androidx.appcompat:appcompat:1.0.2"
implementation "androidx.recyclerview:recyclerview:1.0.0"
implementation "androidx.cardview:cardview:1.0.0"
implementation "androidx.gridlayout:gridlayout:1.0.0"
implementation "androidx.constraintlayout:constraintlayout:1.1.3"
implementation "androidx.lifecycle:lifecycle-process:2.2.0"
implementation "android.arch.work:work-runtime-ktx:$workManagerVersion"
implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlinVersion"
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlinVersion"
implementation "org.jetbrains.kotlin:kotlin-reflect:$kotlinVersion"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core:$kotlinCoroutinesVersion"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:$kotlinCoroutinesVersion"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-rx2:$kotlinCoroutinesVersion"
implementation "io.reactivex.rxjava2:rxkotlin:2.4.0"
implementation("io.reactivex.rxjava2:rxandroid:2.1.1") {
exclude group: "io.reactivex.rxjava2", module: "rxjava"
}
implementation "com.squareup.retrofit2:retrofit:$retrofitVersion"
implementation "com.squareup.retrofit2:converter-jackson:$retrofitVersion"
implementation("com.squareup.retrofit2:converter-simplexml:$retrofitVersion") {
exclude group: "stax", module: "stax-api"
exclude group: "stax", module: "stax"
exclude group: "xpp3", module: "xpp3"
}
implementation "com.squareup.retrofit2:adapter-rxjava2:$retrofitVersion"
implementation "com.squareup.okhttp3:logging-interceptor:$okHttpVersion"
implementation "com.fasterxml.jackson.module:jackson-module-kotlin:2.9.6"
implementation "com.clover.sdk:clover-android-sdk:$cloverVersion"
implementation "com.clover.sdk:clover-android-connector-sdk:$cloverVersion"
implementation "com.google.dagger:dagger:$daggerVersion"
implementation "org.slf4j:slf4j-api:$slf4jVersion"
implementation "com.amazonaws:aws-android-sdk-core:$awsSdkVersion"
implementation "com.amazonaws:aws-android-sdk-pinpoint:$awsSdkVersion"
implementation("com.amazonaws:aws-android-sdk-mobile-client:$awsSdkVersion@aar") {
transitive = true
}
implementation "com.amazonaws:aws-android-sdk-s3:$awsSdkVersion"
implementation "com.amazonaws:aws-android-sdk-appsync:$awsAppSyncSdkVersion"
implementation "org.eclipse.paho:org.eclipse.paho.client.mqttv3:1.2.0"
implementation "org.eclipse.paho:org.eclipse.paho.android.service:1.1.1"
implementation "org.greenrobot:eventbus:3.1.1"
implementation "com.github.bumptech.glide:glide:$glideVersion"
implementation "com.github.bumptech.glide:okhttp3-integration:$glideVersion"
implementation("com.madgag.spongycastle:prov:1.58.0.0") {
exclude group: "junit"
}
implementation "com.auth0.android:jwtdecode:1.4.0"
implementation "net.grandcentrix.tray:tray:0.12.0"
implementation "org.threeten:threetenbp:1.4.1"
implementation("com.jakewharton.threetenabp:threetenabp:1.2.2") {
exclude module: "threetenbp"
}
implementation("com.github.joschi.jackson:jackson-datatype-threetenbp:2.8.4") {
exclude module: "threetenbp"
}
implementation "com.newrelic.agent.android:android-agent:$newRelicVersion"
implementation "com.journeyapps:zxing-android-embedded:3.6.0"
implementation "com.g00fy2:versioncompare:1.3.4"
implementation "com.airbnb.android:lottie:3.3.1"
implementation project(":app:libs:clover:roam")
implementation("com.firstdata.clovergo:remote-pay-android-go-connector:3.3.1.4@aar") {
transitive = true
}
implementation('io.branch.sdk.android:library:4.3.2') {
exclude module: "bbpos"
}
debugImplementation "com.squareup.leakcanary:leakcanary-android:$leakCanaryVersion"
compileOnly "org.glassfish:javax.annotation:10.0-b28"
runtimeOnly "com.noveogroup.android:android-logger:$androidLoggerVersion"
testImplementation "org.slf4j:slf4j-simple:$slf4jVersion"
testImplementation "org.slf4j:jcl-over-slf4j:$slf4jVersion"
testImplementation "org.hamcrest:hamcrest-junit:2.0.0.0"
testImplementation("org.mockito:mockito-core:$mockitoVersion") {
exclude group: "net.bytebuddy"
}
testImplementation("com.nhaarman.mockitokotlin2:mockito-kotlin:$mockitoKotlinVersion") {
exclude group: "org.jetbrains.kotlin"
exclude group: "org.mockito"
}
testImplementation "net.bytebuddy:byte-buddy-android:$byteBuddyVersion"
testImplementation "net.bytebuddy:byte-buddy-agent:$byteBuddyVersion"
testImplementation "org.jetbrains.kotlinx:kotlinx-coroutines-test:$kotlinCoroutinesVersion"
androidTestImplementation "androidx.test:runner:1.2.0"
androidTestImplementation "androidx.test:rules:1.2.0"
androidTestImplementation "androidx.test.uiautomator:uiautomator:2.2.0"
androidTestImplementation "androidx.multidex:multidex-instrumentation:2.0.0"
androidTestImplementation "androidx.test.espresso:espresso-core:$espressoVersion"
androidTestImplementation("androidx.test.espresso:espresso-contrib:$espressoVersion") {
exclude module: "material"
exclude module: "appcompat"
exclude module: "recyclerview"
exclude module: "cardview"
}
androidTestImplementation "androidx.test.espresso:espresso-intents:$espressoVersion"
androidTestImplementation "android.arch.work:work-testing:$workManagerVersion"
androidTestImplementation "com.google.dagger:dagger:$daggerVersion"
androidTestImplementation "org.awaitility:awaitility:3.1.6"
androidTestImplementation "com.squareup.okhttp3:mockwebserver:$okHttpVersion"
androidTestImplementation("org.mockito:mockito-android:$mockitoVersion") {
exclude group: "net.bytebuddy"
}
androidTestImplementation("com.nhaarman.mockitokotlin2:mockito-kotlin:$mockitoKotlinVersion") {
exclude group: "org.jetbrains.kotlin"
exclude group: "org.mockito"
}
androidTestImplementation "net.bytebuddy:byte-buddy-android:$byteBuddyVersion"
androidTestImplementation "net.bytebuddy:byte-buddy-agent:$byteBuddyVersion"
androidTestImplementation "com.squareup.spoon:spoon-client:1.7.1"
androidTestImplementation "org.jetbrains.kotlinx:kotlinx-coroutines-test:$kotlinCoroutinesVersion"
}
android.unitTestVariants.all { variant ->
variant.getRuntimeConfiguration().exclude group: "com.noveogroup.android", module: "android-logger"
}
Okay so this was a fun one, leaving this up incase anyone ever runs into a similar issue.
It seems in this case, the error message we were getting out was fairly misleading. A good way to help diagnose these sort of errors is to look at the tombstone left by the crash, see https://source.android.com/devices/tech/debug/native-crash for more info around that
In this case proguard was our enemy, it seemed to be performing some sort of optimisation on the test code leading to variables being assigned incorrectly and was resolved by adding -optimizations *other optimizations*,!code/allocation/variable this might not work for your particular case, but maybe try configuring proguard to do no optimisation and see if that helps :D