How to let the PSOldGen expand to accommodate very large objects?

Question

I have a program which needs to allocate heap space for one very large object, but I'm finding that I get OutOfMemoryExceptions when there appears to be much more free heap than that large object should require.

This test program demonstrates the problem:

public class HeapTest {
  public static void main(String[] args) {
    final int size = Integer.parseInt(args[0]);
    System.out.println(size >> 20);
    final byte[] buf = new byte[size];
    System.out.println(buf.length);
  }
}

If I run it as java -Xmx40M -XX:+PrintGCDetails HeapTest 28000000, I get the following output:

26
[GC [PSYoungGen: 317K->176K(9216K)] 317K->176K(36544K), 0.0007430 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
[GC [PSYoungGen: 176K->144K(9216K)] 176K->144K(36544K), 0.0004650 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
[Full GC [PSYoungGen: 144K->0K(9216K)] [PSOldGen: 0K->115K(9216K)] 144K->115K(18432K) [PSPermGen: 2710K->2710K(21248K)], 0.0053960 secs] [Times: user=0.00 sys=0.00, real=0.01 secs] 
[GC [PSYoungGen: 0K->0K(9216K)] 115K->115K(36544K), 0.0002010 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
[Full GC [PSYoungGen: 0K->0K(9216K)] [PSOldGen: 115K->112K(15104K)] 115K->112K(24320K) [PSPermGen: 2710K->2708K(21248K)], 0.0078150 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at HeapTest.main(HeapTest.java:5)
Heap
 PSYoungGen      total 9216K, used 476K [0x00000000ff2b0000, 0x0000000100000000, 0x0000000100000000)
  eden space 7936K, 6% used [0x00000000ff2b0000,0x00000000ff327190,0x00000000ffa70000)
  from space 1280K, 0% used [0x00000000ffa70000,0x00000000ffa70000,0x00000000ffbb0000)
  to   space 1280K, 0% used [0x00000000ffec0000,0x00000000ffec0000,0x0000000100000000)
 PSOldGen        total 27328K, used 112K [0x00000000fd800000, 0x00000000ff2b0000, 0x00000000ff2b0000)
  object space 27328K, 0% used [0x00000000fd800000,0x00000000fd81c378,0x00000000ff2b0000)
 PSPermGen       total 21248K, used 2815K [0x00000000f3000000, 0x00000000f44c0000, 0x00000000fd800000)
  object space 21248K, 13% used [0x00000000f3000000,0x00000000f32bff48,0x00000000f44c0000)

The array I'm trying to create should be about 27344K, which slightly exceeds the size of the PSOldGen object space. However, the total unused space is about 54000K, which is almost twice what's needed for my array. Since my (real) program works on the array in-place, it uses barely any memory beyond what's required for the array, so allocating twice the memory I need and using only half of it seems wasteful.

Is there a way that I can convince the JVM to let the old generation to grow more than it seems able to by default?

Answer 1

You can use NewRatio, NewSize, MaxNewSize and SurvivorRatio to tune the size of Young Generation, thus indirectly also tuning the size of Old Generation.

Also, if you are worried about Permanent Generation using too much memory, you can tune that with MaxPermSize command line option.

See Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine for more details, especially the part about generations.

For example, setting the maximum Young Generation size to 10M allows your sample program to run successfully:

java -Xmx40M -XX:+PrintGCDetails -XX:MaxNewSize=10M HeapTest 28000000

produces the following output on my machine:

26
28000000
Heap
 par new generation   total 9216K, used 1499K [106810000, 107210000, 107210000)
  eden space 8192K,  18% used [106810000, 106986ed0, 107010000)
  from space 1024K,   0% used [107010000, 107010000, 107110000)
  to   space 1024K,   0% used [107110000, 107110000, 107210000)
 concurrent mark-sweep generation total 30720K, used 27343K [107210000, 109010000, 109010000)
 concurrent-mark-sweep perm gen total 21248K, used 4501K [109010000, 10a4d0000, 10e410000)