Hung Glassfish after limited usage, GC?

I'm working on the following setup:

OS         CentOS release 6.4 (Final)
Memory     1024MB
CPU        1 × 2.4 Ghz @ 80%
Glassfish  GlassFish Server Open Source Edition 3.1.2.2 (build 5)

I'm aware that this is only a small setup with limited memory, though it should suffice.

Here's my problem (I know, there's written a lot about it already):

After some usage my memory seems to clog up. This leads to a hung Glassfish, meminfo show something like:

MemTotal:        1030772 kB
MemFree:          158488 kB
Buffers:            3204 kB
Cached:            16340 kB
SwapCached:         7100 kB
Active:           413424 kB
Inactive:         410252 kB

top shows:

PID  USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
2380 root      20   0  860m 658m 6028 S 99.8 65.4 170:58.30 java

Conclusion

My Glassfish server tends to use all resources after a while. I'm not sure as too why the CPU is clogged, though I'm suspecting this has something to do with Garbage Collection.

My question is, how can I prevent this from happening? Should I configure my GC, if so, how?

Solution

1024MB isn't trivial. I don't know what your app is doing, but it's a respectable start.

Since Java uses a generational memory model, I'd recommend getting a dynamic picture of all the generations: perm gen, eden, etc.

I like Visual VM, with all the plug-ins installed. It lets me see memory, threads, CPU, objects in real time. Try it and see if it'll help you. More information is what you need.