Why is JVM continously doing full gc while the Old Generation is only half full?

Question

I am using jdk 1.7.0_09 on a CentOS 64-bit linux machine.

The gc related vm args is

-Xmx4g -Xmn2g -XX:SurvivorRatio=4 -XX:PermSize=128m -XX:MaxPermSize=128m -XX:InitialTenuringThreshold=15 -XX:CMSWaitDuration=50 -XX:MaxTenuringThreshold=15 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=80 -XX:+CMSParallelRemarkEnabled -XX:ReservedCodeCacheSize=128m

But it's continuously doing full gc

jstat -gcutil pid 1000    
    S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT   
    0.00   0.00  15.62  57.52  19.90     11    2.081 11109 7454.686 7456.767   
    0.00   0.00  15.81  57.52  19.90     11    2.081 11109 7454.686 7456.767   
    0.00   0.00  15.81  57.51  19.90     11    2.081 11111 7454.892 7456.973   
    0.00   0.00  16.06  57.51  19.90     11    2.081 11111 7455.115 7457.196
    0.00   0.00  16.06  57.51  19.90     11    2.081 11111 7455.115 7457.196
    0.00   0.00  16.27  57.51  19.90     11    2.081 11111 7455.115 7457.196
    0.00   0.00  16.27  57.51  19.90     11    2.081 11111 7455.115 7457.196
    0.00   0.00  16.29  57.51  19.90     11    2.081 11111 7455.115 7457.196
    0.00   0.00  16.29  57.51  19.90     11    2.081 11111 7455.115 7457.196
    0.00   0.00  16.29  57.47  19.90     11    2.081 11113 7455.549 7457.629
    0.00   0.00  16.29  57.47  19.90     11    2.081 11113 7455.549 7457.629
    0.00   0.00  16.34  57.47  19.90     11    2.081 11113 7455.549 7457.629
    0.00   0.00  16.34  57.47  19.90     11    2.081 11113 7455.549 7457.629
    0.00   0.00  16.34  57.47  19.90     11    2.081 11113 7455.549 7457.629
    0.00   0.00  16.42  57.47  19.90     11    2.081 11114 7455.549 7457.629
    0.00   0.00  16.42  57.44  19.90     11    2.081 11115 7455.986 7458.067

Answer 1

Despite the column "FGC", it is not really a full GC. What the CMS is doing is a concurrent collection to see if anything can be cleaned up from the old collection (which you can see it is doing). The way to can tell this not a full GC, is that a full GC clears the eden space and copies from one survivor space to another.

IMHO The more options you use, the more likely you will come across some combination with surprising results. I would stick to the minimum of options and see how that performs. Only add option when you know they will always help.

Try to start with,

-mx4g -XX:MaxPermSize=128m -XX:+UseConcMarkSweepGC