Is there a way howto instruct GO runtime to use larger heaps? I am running GO 1.5.
My GO process is currently spending 34% of time in GC but it uses unly 1/3 of available systems memory.
I know ulimit can be used to limit max heap size. I have set ulimit to ~16GB (ulimit -v 17179869184) but the heap size never goes over 5GB.
Using GODEBUG=gctrace=1 I can see high GC overhead (34%):
20160719-220359.169294 :: gc 665 @5484.983s 34%: 3.3+2504+188+1635+8.0 ms clock, 26+2504+0+26950/3271/3.5+64 ms cpu, 4825->4964->2623 MB, 4948 MB goal, 8 P
20160719-220406.322354 :: gc 666 @5492.411s 34%: 2.9+212+2111+1749+8.3 ms clock, 23+212+0+25010/3496/146+67 ms cpu, 4846->4990->2657 MB, 4970 MB goal, 8 P
20160719-220413.703514 :: gc 667 @5499.452s 34%: 4.4+4411+0.021+0.25+8.4 ms clock, 35+4411+0+29365/0.054/38+67 ms cpu, 4908->5022->2618 MB, 5025 MB goal, 8 P
You can control this with the GOGC environment variable.
It is a percentage: set it to 200 and the Go runtime will use twice as much memory as before.
[this was buried in the comments; I'm making it visible as an answer]
Update: there is a detailed discussion of different techniques at https://github.com/golang/go/issues/23044, including mention of the "ballast" technique.