How to serialize/deserialize data to file in Rascal MPL

Question

I need a way to serialize data in Rascal as some operations may be relatively slow and some quick form of caching is desired. For example the construction of ASTs.

I'm using the following segment of code to construct ASTs from the project at the indicated location:

list[Declaration] getASTs(loc projectLocation) {
    M3 model = createM3FromMavenProject(projectLocation);
    list[Declaration] asts = [createAstFromFile(f, true)
        | f <- files(model.containment), isCompilationUnit(f)];
    return asts;
}

Now constructing a particular AST takes some time (around 2-3 seconds). I therefore wish to cache the result inside some dump file:

loc smallsqlLoc = projectLoc + "smallsql0.21_src";
loc dumpLoc = projectLoc + "dump/smallsql.bin";

if(!exists(dumpLoc)) {
    list[Declaration] dumpAsts = getASTs(smallsqlLoc);
    writeFile(dumpLoc, dumpAsts);
}

And read it back into memory (which hopefully will be way faster):

if(!exists(dumpLoc))
    throw "Error: dump does not exist.";
list[Declaration] asts = readFile(dumpLoc);

But the return type of readFile is str, not list[Declaration]. A simple cast will obviously not solve the problem. How can I get the AST I originally wrote to the file back into memory? More generally, how to serialize and deserialize any data type from/to a file in Rascal?

Answer 1

You are looking for writeBinaryValueFile (and the readBinaryValueFile) they takes care of reading and writing almost any rascal value (the exception is functions, we cannot serialize them).

The writeFile you are using is for text files, but those are not meant for data storage (unless you are generating source code offcourse).

usage example:

model = createM3FromMavenProject(projectLocation);
writeBinaryValueFile(|tmp:///test.cache|, model);
model2 = readBinaryValueFile(#M3, |tmp:///test.cache|);

So, regarding caching, if you want cross-runtime caching, writeBinaryValueFile is the way to go. If you need caching inside the same session, you can take a look at the @memo tag, that behaves as a cache. It's not documented that clearly, but you can take a peek at the tests around this feature: lang::rascal::tests::basic::Memoization.