Search code examples
kotlingsonretrofitdeserializationproguard

How to deserialize nested Json API response in Kotlin using Gson @SerializedName annotations


I am migrating a project to a new API and thus have to update the deserialization and data classes. Using the old API and data classes everything works fine, but using the updated version I get the following error

java.lang.ClassNotFoundException: Didn't find class "fooapp.component.ui.cafeteria.model.CafeteriaData" on path: DexPathList[[zip file "/data/app/~~Wji9DkrkAr7qFGIIJ3Aglg==/fooapp-dbYvbr8le25I8FGioIrW6w==/base.apk"],nativeLibraryDirectories=[/data/app/~~Wji9DkrkAr7qFGIIJ3Aglg==/fooapp-dbYvbr8le25I8FGioIrW6w==/lib/x86_64, /system/lib64, /system_ext/lib64]]

when attempting to access a nested field, eg:

response.cafeterias[0]

Note that response.cafeterias is parsed correctly and accessible. So it seems like parsing works for nesting level 1, but somehow breaks when stepping down into the next level.

I am aware that the prices field is missing in the mapping, it is not needed for now. I will check if adding it to the serialization hierarchy fixes the issue tomorrow.

What I have tried so far

  • Invalidating Cache and restarting
  • Cleaning and rebuilding the project
  • Disabling Hot swapping
  • Taking a small sample JSON and parsing it directly instead of letting GSON parse it in the background
  • Both val and var for the fields in the data classes that are involved

Sample JSON

{"canteens": [
    {
        "version": "2.1",
        "canteen_id": "best-canteen",
        "weeks": [
            {
                "number": 48,
                "year": 2021,
                "days": [
                    {
                        "date": "2021-11-29",
                        "dishes": [
                            {
                                "name": "Tasty food 1",
                                "prices": {
                                    "students": {
                                        "base_price": 0,
                                        "price_per_unit": 0.33,
                                        "unit": "100g"
                                    },
                                    "staff": {
                                        "base_price": 0,
                                        "price_per_unit": 0.55,
                                        "unit": "100g"
                                    },
                                    "guests": {
                                        "base_price": 0,
                                        "price_per_unit": 0.66,
                                        "unit": "100g"
                                    }
                                },
                                "ingredients": [
                                    "Gl",
                                    "GlW",
                                    "Kn"
                                ],
                                "dish_type": "Daily 4"
                            }]
                    }]
            }]
    }]
}

Proguard rules

proguard-gson.pro

## GSON 2.2.4 specific rules ##

# Gson uses generic type information stored in a class file when working with fields. Proguard
# removes such information by default, so configure it to keep all of it.
-keepattributes Signature

# For using GSON @Expose annotation
-keepattributes *Annotation*

-keepattributes EnclosingMethod

# Gson specific classes
-keep class sun.misc.Unsafe { *; }
-keep class com.google.gson.stream.** { *; }

proguard-square-okhttp.pro

# OkHttp
-keepattributes Signature
-keepattributes *Annotation*
-keep class com.squareup.okhttp.** { *; }
-keep interface com.squareup.okhttp.** { *; }
-dontwarn com.squareup.okhttp.**

proguard-square-okhttp3.pro

# OkHttp
-keepattributes Signature
-keepattributes *Annotation*
-keep class okhttp3.** { *; }
-keep interface okhttp3.** { *; }
-dontwarn okhttp3.**

proguard-square-retrofit.pro

# Retrofit 1.X

-keep class com.squareup.okhttp.** { *; }
-keep class retrofit.** { *; }
-keep interface com.squareup.okhttp.** { *; }

-dontwarn com.squareup.okhttp.**
-dontwarn okio.**
-dontwarn retrofit.**
-dontwarn rx.**

-keepclasseswithmembers class * {
    @retrofit.http.* <methods>;
}

# If in your rest service interface you use methods with Callback argument.
-keepattributes Exceptions

# If your rest service methods throw custom exceptions, because you've defined an ErrorHandler.
-keepattributes Signature

# Also you must note that if you are using GSON for conversion from JSON to POJO representation, you must ignore those POJO classes from being obfuscated.
# Here include the POJO's that have you have created for mapping JSON response to POJO for example.

proguard-square-retrofit2.pro

# Retrofit 2.X
## https://square.github.io/retrofit/ ##

-dontwarn retrofit2.**
-keep class retrofit2.** { *; }
-keepattributes Signature
-keepattributes Exceptions

-keepclasseswithmembers class * {
    @retrofit2.http.* <methods>;
}

proguard-guava.pro

# Configuration for Guava 18.0
#
# disagrees with instructions provided by Guava project: https://code.google.com/p/guava-libraries/wiki/UsingProGuardWithGuava

-keep class com.google.common.io.Resources {
    public static <methods>;
}
-keep class com.google.common.collect.Lists {
    public static ** reverse(**);
}
-keep class com.google.common.base.Charsets {
    public static <fields>;
}

-keep class com.google.common.base.Joiner {
    public static com.google.common.base.Joiner on(java.lang.String);
    public ** join(...);
}

-keep class com.google.common.collect.MapMakerInternalMap$ReferenceEntry
-keep class com.google.common.cache.LocalCache$ReferenceEntry

# http://stackoverflow.com/questions/9120338/proguard-configuration-for-guava-with-obfuscation-and-optimization
-dontwarn javax.annotation.**
-dontwarn javax.inject.**
-dontwarn sun.misc.Unsafe

# Guava 19.0
-dontwarn java.lang.ClassValue
-dontwarn com.google.j2objc.annotations.Weak
-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement

# Guava 20.0
-dontwarn com.google.**

# Guava 23.5
-dontwarn afu.org.checkerframework.**
-dontwarn org.checkerframework.** 

proguard-project-rules.pro

# Add project specific ProGuard rules here.
#
# For more details, see
#   http://developer.android.com/guide/developing/tools/proguard.html

-dontobfuscate

-keep class fooapp.models.**{*;}

proguard-simple-xml.pro

# Simple-Xml Proguard Config
# NOTE: You should also include the Android Proguard config found with the build tools:
# $ANDROID_HOME/tools/proguard/proguard-android.txt

# Keep public classes and methods.
-dontwarn com.bea.xml.stream.**
-dontwarn org.simpleframework.xml.stream.**
-keep class org.simpleframework.xml.**{ *; }
-keepclassmembers,allowobfuscation class * {
    @org.simpleframework.xml.* <fields>;
    @org.simpleframework.xml.* <init>(...);
}

proguard-sqlite.pro

-keep class org.sqlite.** { *; }
-keep class org.sqlite.database.** { *; }

proguard-square-okio.pro

# Okio
-keep class sun.misc.Unsafe { *; }
-dontwarn java.nio.file.*
-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement
-dontwarn okio.**

proguard-tikxml.pro

-keep class com.tickaroo.tikxml.** { *; }
-keep class **$$TypeAdapter { *; }

-keepclasseswithmembernames class * {
    @com.tickaroo.tikxml.* <fields>;
}

-keepclasseswithmembernames class * {
    @com.tickaroo.tikxml.* <methods>;
}

Data classes

CafeteriaResponse

data class CafeteriaResponse(
    @SerializedName("canteens")
    var cafeterias: List<CafeteriaData>,
)

CafeteriaData

data class CafeteriaData(
        @SerializedName("version")
        var version: String? = null,
        @SerializedName("canteen_id")
        var cafeteriaSlug: String? = null,
        @SerializedName("weeks")
        var menusByWeeks: List<WeeklyMenu>
)

WeeklyMenu

data class WeeklyMenu(
        @SerializedName("number")
        var weekOfYear: Short = -1,
        @SerializedName("year")
        var year: Short = -1,
        @SerializedName("days")
        var dishesForWeek: List<DailyMenu>
)

DailyMenu

data class DailyMenu(
        @SerializedName("date")
        var date: DateTime? = null,
        @SerializedName("dishes")
        var dishesForDay: List<Dish>
)

Dish

data class Dish(
        @SerializedName("name")
        var name: String? = null,
        @SerializedName("ingredients")
        var ingredients: List<String>,
        @SerializedName("dish_type")
        var type: String? = null
)

All of these data classes are in the same package.


Solution

  • Problem

    proguard-square-retrofit.pro needed to be updated including the new POJOs that are used for the API data deserialization.

    Solution

    I added all the POJOs to the below proguard ruleset. I put them all in a seperate package to get finer granularity in obfuscation and avoid keeping unnecessary classes.

    Updated proguard-square-retrofit.pro

    # Retrofit 1.X
    
    -keep class com.squareup.okhttp.** { *; }
    -keep class retrofit.** { *; }
    -keep interface com.squareup.okhttp.** { *; }
    
    -dontwarn com.squareup.okhttp.**
    -dontwarn okio.**
    -dontwarn retrofit.**
    -dontwarn rx.**
    
    -keepclasseswithmembers class * {
        @retrofit.http.* <methods>;
    }
    
    # If in your rest service interface you use methods with Callback argument.
    -keepattributes Exceptions
    
    # If your rest service methods throw custom exceptions, because you've defined an ErrorHandler.
    -keepattributes Signature
    
    # Also you must note that if you are using GSON for conversion from JSON to POJO representation, you must ignore those POJO classes from being obfuscated.
    # Here include the POJO's that have you have created for mapping JSON response to POJO for example.
    -keep class fooapp.component.ui.cafeteria.model.deserialization.*