How to unpickle a file in Java?

Question

I have a 'pickle' format datafile that was created by a Python process (actually, the MCDungeon cache file). I want to read this from a Java program.

To do this, I have used this code:

    public HashMap<String, Object> getDataFileStream(String filename) {
        HashMap<String, Object> data = new HashMap<String, Object>();
        File f = new File(filename);
        InputStream fs = null;
        try {
            fs = new FileInputStream(f);
        } catch (FileNotFoundException e) {
            Log.warning("Pickle file '"+filename+"' not found!");
            return null;
        }

        PyFile picklefile = new PyFile(fs);    
        PyDictionary phash = null;
        try {
            phash = (PyDictionary) cPickle.load(picklefile);        
        } catch ( PyException e3 ) {
            log.severe("Cannot unpickle! (Python error)");
            e3.printStackTrace(); // throws a null pointer exception
            return null;    
        } catch ( Exception e ) {
            log.severe("Cannot unpickle! Err: " + e.getClass().getName());
            e.printStackTrace();
            return null;
        }
    ConcurrentMap<PyObject, PyObject> aMap = phash.getMap();
    for (Map.Entry<PyObject, PyObject> entry : aMap.entrySet()) {
        String keyval = entry.getKey().toString();
        PyObject tileentity = (PyList) entry.getValue();
        try {
            data.put(keyval, pythonToJava(tileentity));
        } catch (Exception e) {
            data.put(keyval, tileentity);
        }
    }
    return data;
}

I have included the JYthon library, and also the pythonToJava function (elsewhere).

The file I am passing is definitely a valid Picklefile as it can be read by the Python process successfully.

However, when running this function, I get a PyException thrown in the cPickle.load function, and calling the printStackTrace given a NullPointer exception (the line number 71 corresponds to the e3.printStackTrace() line above)

[12:42:18 ERROR]: [DynmapMCDungeon] Cannot unpickle! (Python error)
java.lang.NullPointerException
        at org.steveshipway.dynmap.PickleLoader.getDataFileStream(PickleLoader.j
ava:71) ~[?:?]
        at org.steveshipway.dynmap.Dungeon.getDungeons(Dungeon.java:28) ~[?:?]
        at org.steveshipway.dynmap.DynmapMCDungeon.activate(DynmapMCDungeon.java
:179) ~[?:?]

When I load the Pickle data in manually and pass to the function, I get a NullPointerException error in the cPickle.load function:

[13:56:57 INFO]: [DynmapMCDungeon] Reading in MCDungeon pickle...
[13:56:57 ERROR]: [DynmapMCDungeon] Cannot unpickle the MCDungeon cache! Err: java.lang.NullPointerException
[13:56:57 WARN]: java.lang.NullPointerException
[13:56:57 WARN]:        at java.util.Objects.requireNonNull(Unknown Source)
[13:56:57 WARN]:        at java.util.Arrays$ArrayList.<init>(Unknown Source)
[13:56:57 WARN]:        at java.util.Arrays.asList(Unknown Source)
[13:56:57 WARN]:        at org.python.core.PyList.<init>(PyList.java:52)
[13:56:57 WARN]:        at org.python.core.PyList.<init>(PyList.java:64)
[13:56:57 WARN]:        at org.python.modules.cPickle$Unpickler.load_empty_list(cPickle.java:1909)
[13:56:57 WARN]:        at org.python.modules.cPickle$Unpickler.load(cPickle.java:1620)
[13:56:57 WARN]:        at org.python.modules.cPickle.load(cPickle.java:636)
[13:56:57 WARN]:        at org.steveshipway.dynmap.PickleLoader.getDataFileStream(PickleLoader.java:64)

My questions are:

1. Why am I getting an error when I try to print the stack trace?

2. What am I doing wrong with loading the picklefile? Is there a better way to achieve this?

Thanks in advance for any pointers (preferably not null ones, I have enough of those already!)

Answer 1

I successfully unpickled a file using https://github.com/irmen/pickle - Pickle protocol version support:

• reading: 0,1,2,3,4
• writing: 2

Does not require Jython.