Search code examples
javajsonmemory-leaksout-of-memorytomcat8

JSON Builder cause memory leak in Tomcat8 Application

I've created a WebApp for a Tomcat8 Server. It is a small webbased game.

Since I've switched from flatfiles to MySQL I have a lot of problems with javax.json builder.

Here the part of my code:

public void saveUserResearch(){
    MySQLTable t = SpaceWar.instance().getUserResearchTable();

    JsonObjectBuilder mainBuilder = Json.createObjectBuilder();
    JsonObjectBuilder subBuilder = Json.createObjectBuilder();
    for(UserResearch r : getResearchList()){
        int index = r.getIndex();
        subBuilder = Json.createObjectBuilder();
        subBuilder.add("index", r.getIndex());
        subBuilder.add("level", r.getLevel());
        subBuilder.add("planed_level", r.getPlanedLevel());
        subBuilder.add("player_uuid", r.getPlayerUUIDAsString());
        subBuilder.add("unlocked", r.getUnlocked());
        subBuilder.add("savedInPlayer", "yes");
        mainBuilder.add(index+"", subBuilder.build());
    }
    String q = "";
    q += "user_uuid:" + getUUID().toString() + ";";
    q += "research_json:" + mainBuilder.build();
    t.insertUpdate(q).sync();

}

Maven Dependencies:

<!-- https://mvnrepository.com/artifact/javax.json/javax.json-api -->
        <dependency>
            <groupId>javax.json</groupId>
            <artifactId>javax.json-api</artifactId>
            <version>1.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.glassfish/javax.json -->
        <dependency>
            <groupId>org.glassfish</groupId>
            <artifactId>javax.json</artifactId>
            <version>1.0.4</version>
        </dependency>

This method I am calling from a Thread every 2 minutes. I have 15 users in the game, so it runs 15 times. It creates a JSON Object and the JSON object I am saving to a MySQL DB. So I am expecting 15 instances.

The problem is after a few hours, the Tomcat Server crashes with following message.

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000007b5400000, 1048576, 0) failed; error='Cannot allocate memory' (errno=12)

 There is insufficient memory for the Java Runtime Environment to continue.
 Native memory allocation (mmap) failed to map 1048576 bytes for committing reserved memory.
 An error report file with more information is saved as:
 /tmp/hs_err_pid11069.log

So I decided to use VisualVM to take a look into it and there I've found a lot of trouble.

enter image description here enter image description here

In the first image you can see that there are the same user json object.

So I tried to set the mainBuilder and subBuilder to null. But with the same result. Also I tried to call System.gc(); but it does not clear the stuff.

After 2h it looks like this

enter image description here enter image description here

And after about 5 hours it runs out of memory

These are my JVM Arguments enter image description here

I am not sure why the builder cause this kind of memory leak and my knowledge is not great enough to solve it. Maybe you can help me.

I would be very grateful. Thank you very much.

For the MySQL stuff I am using an API from here: SpiderMySQL And the javax.json.Json; javax.json.JsonObjectBuilder;

Solution

  • I think i found the issue. It is the mysql lib! look at the class here: https://gitlab.com/Syranda/SpiderMySQL/blob/development/src/main/java/de/syranda/spidermysql/WatchThread.java

    It is not thread safe, since ArrayList is not thread safe. They have already a bug ticket for this.

    In combination of this class: https://gitlab.com/Syranda/SpiderMySQL/blob/development/src/main/java/de/syranda/spidermysql/table/InsertQuery.java

    It lets the this reference escape in the consturctor. Which is very evil in multithreaded environments. It can produce strange behavior and maybe your memory leak.

    you can try to switch to pure jdbc in that case just to test if the memory leak stays. And i recommend to use any other lib, it seems not to be implemented really well.

    I tested the JSON impl, it has no memory leak