The VMOptions currently we using are like below for one of our microservice which designed with spring webflux. We currently have 4 CPU and 5 GB of memory reserved for each pod.
-Xms4096m -Xmx4096m -Xss512k -XX:+UseG1GC -XX:+UseStringDeduplication -XX:ParallelGCThreads=8 -XX:ConcGCThreads=2 -XX:MaxGCPauseMillis=120 -XX:G1ReservePercent=10
Every time I fresh deploy our App and introduce some fixed QPS load, I observed there are 3 large spikes occur on OldGen then, it managed to go in some stable harmony. I was expecting stable harmony from the very beginning but not sure about the behaviour coming at the beginning. Can anybody also faced such a situation or know the fix.
You observe the effect of G1 Adaptive IHOP.
From the G1 tuning guide:
The Initiating Heap Occupancy Percent (IHOP) is the threshold at which an Initial Mark collection is triggered and it is defined as a percentage of the old generation size.
G1 by default automatically determines an optimal IHOP by observing how long marking takes and how much memory is typically allocated in the old generation during marking cycles. This feature is called Adaptive IHOP. If this feature is active, then the option -XX:InitiatingHeapOccupancyPercent determines the initial value as a percentage of the size of the current old generation as long as there aren't enough observations to make a good prediction of the Initiating Heap Occupancy threshold. Turn off this behavior of G1 using the option -XX:-G1UseAdaptiveIHOP. In this case, the value of -XX:InitiatingHeapOccupancyPercent always determines this threshold.
By default, -XX:G1AdaptiveIHOPNumInitialSamples
is equal to 3. This means, G1 uses the first 3 completed marking cycles as input for prediction of the optimal IHOP.
There is nothing wrong in such behavior, no need to "fix" anything. But if you know the optimal IHOP value for your particular application, and want GC to use it from the very beginning, turn off -XX:-G1UseAdaptiveIHOP
, and set -XX:InitiatingHeapOccupancyPercent
manually.