Search code examples
javaserializationkryo

Storing a kryo object in a compiled jar?


I have a HashMap I'd like to persist and have fast access to. I use Kryo to serialize the object

    Kryo kryo = new Kryo();
    MapSerializer serializer = new MapSerializer();
    kryo.register(Location.class);
    kryo.register(HashMap.class, serializer);

    Output output = new Output(new FileOutputStream("src/main/resources/locations50K.kryo"));
    kryo.writeObject(output, locationMap);
    output.close();

I can successfully deserialize with

    Input input = new Input(new FileInputStream("src/main/resources/locations50K.kryo"));      
    Map<String, Location> locationMap;
    locationMap = kryo.readObject(input, HashMap.class);
    input.close();

    log.info(locationMap.size());

the log.info shows that I have 231,045 entries in my map.

Now, I would like to access my .kryo file after I compiled a *-jar-with-dependencies.jar (I'm using Maven). So, rather than a FileInputStream that reads from src/main/resources/, I use MyClass.class.getResourceAsStream

    InputStream isr = MyClass.class.getResourceAsStream("/locations50K.kryo");

    if(isr == null)
        log.error("null input");

    Input input = new Input(isr);
    locationMap = kryo.readObject(input, HashMap.class);
    input.close();

    log.info(locationMap.size());

the log.error never shows and the log.info says I have 0 entries in my map. Why? isr isn't null so it is reading something, Kryo just can't seem to deserialize it and doesn't provide any error.


Solution

  • Turns out the problem was with Maven. I had "filtering" enabled so Maven was trying to utf8 encode my serialized objects in the jar. Kryo was silent about it but rewriting the code to use standard Java serialization gave the error found here: https://stackoverflow.com/a/5421992/424631

    Here is the fix:

        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <!--if true Maven will try to UTF-8 encode objects, which breaks deserialization-->
                <filtering>false</filtering>
            </resource>
        </resources>