23 How to optimize JVM memory allocation #

Hello, I am Liu Chao.

JVM tuning is a systematic and complex process, but we know that in most cases, we don’t need to adjust JVM memory allocation because some initial parameters can ensure the normal and stable operation of the application service.

However, all tuning is goal-oriented, and the optimization of JVM memory allocation is no exception. When there is no performance problem, we naturally won’t change the parameters of JVM memory allocation casually. But what about when there is a problem? What kind of performance problem do we need to optimize, and how can we optimize it? That is what I am going to share today.

JVM Memory Allocation Performance Issues #

When it comes to performance issues exhibited by JVM memory, you may think of some online JVM memory overflow accidents. However, such accidents are often caused by the difficulty of memory reclamation due to object creation in the application program, and generally belong to coding problems.

But actually, in many cases, the performance impact caused by improper JVM memory allocation in specific scenarios of application services may not be as prominent as memory overflow issues. It can be said that if you don’t delve into various performance indicators, it is difficult to detect the hidden performance loss.

The most direct manifestation of improper JVM memory allocation is frequent GC, which can lead to performance issues such as context switching, thereby reducing system throughput and increasing system response time. Therefore, if you find frequent GC in the online environment or performance testing, and it is normal object creation and reclamation, it is necessary to consider adjusting JVM memory allocation at this time in order to reduce the performance overhead caused by GC.

Object Survival Cycle in the Heap #

After understanding the performance issues, the next step is undoubtedly optimization. But before diving into the process of optimizing JVM memory allocation, let’s first take a look at the object survival cycle in the heap, which will lay the foundation for our future learning.

In [Lesson 20], I talked about the JVM memory model. As we know, the heap in the JVM memory model is divided into the young generation and the old generation. The young generation is further divided into the Eden area and the Survivor area. Finally, the Survivor area is divided into the From Survivor and To Survivor areas.

When we create a new object, it is allocated into the Eden area of the young generation by default. At this time, the JVM assigns an object age counter to the object (configured by the -XX:MaxTenuringThreshold parameter).

There is also another case where, when the Eden space is insufficient, the JVM will perform a garbage collection in the young generation (Minor GC). At this time, the JVM moves the surviving objects to the Survivor area and increments their ages by 1. Objects in the Survivor area also undergo Minor GC, and each time an object goes through a Minor GC, its age is incremented by 1.

Of course, there is also a threshold for the memory space. By setting the -XX:PetenureSizeThreshold parameter, we can directly allocate the maximum object size that will be allocated into the old generation. If the allocated object exceeds the set threshold, it will be directly allocated into the old generation. This approach can reduce the frequency of garbage collection in the young generation.

Viewing JVM Heap Memory Allocation #

We have learned about the process of an object being created in the heap and then being recycled. Now let’s understand how JVM heap memory is allocated. In the case where the JVM heap memory size is not explicitly configured, the JVM configures the current memory size based on default values. We can use the following commands to view the default values for heap memory configuration:

java -XX:+PrintFlagsFinal -version | grep HeapSize 
jmap -heap 17284

Through these commands, we can obtain that the default maximum heap memory size set by the JVM on this machine is 1953MB, and the initial size is 124MB.

In JDK 1.7, by default, the ratio of the young generation to the old generation is 1:2. We can reset this configuration using the -XX:NewRatio option. The ratio of Eden to Survivor (To and From) in the young generation is 8:1:1. We can reset this configuration using the -XX:SurvivorRatio option.

In JDK 1.7, if the -XX:+UseAdaptiveSizePolicy option is enabled, the JVM will dynamically adjust the sizes of various regions in the Java heap as well as the age at which objects are promoted to the old generation. The -XX:NewRatio and -XX:SurvivorRatio options will become ineffective. However, in JDK 1.8, the -XX:+UseAdaptiveSizePolicy option is enabled by default.

Furthermore, in JDK 1.8, it is not recommended to disable the UseAdaptiveSizePolicy option unless you have a clear plan for the initial heap memory/max heap memory, young/old generation, and Eden/Survivor regions. Otherwise, the JVM will allocate the minimum heap memory, and the young generation and old generation will be allocated in a 1:2 ratio by default, while the Eden and Survivor regions in the young generation will be allocated in an 8:2 ratio by default. This memory allocation may not be the best configuration for your application service and may cause serious performance issues.

Optimization Process of JVM Memory Allocation #

First, we observe the running status of the application service using the default configuration of JVM. Here, I will use an actual case as an example. Let’s simulate a panic-buying interface, assuming that we need to meet a concurrency of 50,000 requests and each request will generate a 20KB object. We can simulate the scenario of generating a large number of objects with a concurrency of 1,000 and a 1MB object creation interface. The specific code is as follows:

@RequestMapping(value = "/test1")
public String test1(HttpServletRequest request) {
    List<Byte[]> temp = new ArrayList<Byte[]>();

    Byte[] b = new Byte[1024*1024];
    temp.add(b);

    return "success";
}

AB Stress Testing #

We perform stress testing on the application service. The following shows the changes in throughput and response time of the request interface under different concurrent user numbers:

We can see that when the concurrency reaches a certain value, the throughput stops increasing and the response time increases rapidly. So, what happens inside the JVM?

Analyzing GC Logs #

At this point, we can check the specific recycling logs through GC logs. We can dump the GC logs during runtime by setting VM configuration parameters. The specific configuration parameters are as follows:

-XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:/log/heapTest.log

The explanations for the configuration items are as follows:

• -XX:PrintGCTimeStamps: prints the specific time of GC;
• -XX:PrintGCDetails: prints out detailed GC logs;
• -Xloggc: path: specifies the path for generating GC logs.

After collecting the GC logs, we can open them using the GCViewer tool introduced in [Lesson 22] to view the specific GC logs:

The main page shows that FullGC occurred 13 times, and the memory utilization rates of the young generation and the old generation were almost 100% in the lower right corner. FullGC will cause stop-the-world, seriously affecting the performance of the application service. At this point, we need to adjust the size of the heap memory to reduce the occurrence of FullGC.

Reference Indicators #

We can use the expected values of certain indicators as reference indicators. The GC frequency mentioned above is one of them. So, what other indicators can provide us with specific optimization directions?

GC Frequency: High-frequency FullGC imposes significant performance overhead on the system. Although MinorGC is much better than FullGC, too much MinorGC still puts pressure on the system.

Memory: Here, memory refers to the size of the heap memory, which is further divided into the young generation memory and the old generation memory. First, we need to analyze whether the size of the heap memory is appropriate, in fact, analyze whether the proportion of the young generation and the old generation is appropriate. Insufficient memory or uneven allocation will increase FullGC, and seriously, it will cause the CPU to continuously run at full load, affecting the system performance.

Throughput: Frequent FullGC causes thread context switching, increasing system performance overhead, thereby affecting the processing of thread requests each time, and ultimately leading to a decrease in system throughput.

Latency: The duration of JVM’s GC will also affect the response time of each request.

Specific Optimization Methods #

Adjust the heap memory space to reduce FullGC occurrence: Through log analysis, we found that the heap memory is almost used up, and there are many FullGCs. This means that our heap memory is severely insufficient. At this time, we need to increase the size of the heap memory.

java -jar -Xms4g -Xmx4g heapTest-0.0.1-SNAPSHOT.jar

The explanations for the configuration items are as follows:

• -Xms: initial size of the heap;
• -Xmx: maximum size of the heap.

After increasing the heap memory size, we test the performance again and find that the throughput has increased by around 40% and the response time has decreased by nearly 50%.

Check the GC logs again and find that the FullGC frequency has decreased, and the utilization rate of the old generation is only 16%.

Adjust the young generation to reduce MinorGC: By adjusting the heap memory size, we have improved the overall throughput and reduced the response time. Is there any more room for optimization? We can set the young generation to be larger, thereby reducing some MinorGCs (Lesson 22 provides a detailed explanation of enhancing system performance by reducing MinorGC frequency).

java -jar -Xms4g -Xmx4g -Xmn3g heapTest-0.0.1-SNAPSHOT.jar

After adjusting the young generation, let’s perform AB stress testing again and find that the throughput has increased.

Check the GC logs again and find that the MinorGC has also significantly reduced, and the total time spent on GC has decreased.

Set the Ratio of Eden and Survivor Regions: In JVM, if AdaptiveSizePolicy is enabled, the sizes of the Eden, From Survivor, and To Survivor regions are recalculated after each GC based on the GC runtime statistics, throughput, and memory usage. At this time, the SurvivorRatio configured by default will become invalid.

In JDK 1.8, AdaptiveSizePolicy is enabled by default. We can disable this configuration by -XX:-UseAdaptiveSizePolicy or explicitly run -XX:SurvivorRatio=8 to set the ratio of Eden and Survivor regions to 8:2. Most new objects are created in the Eden region. By fixing the proportion of the Eden region, we can optimize the memory allocation performance of the JVM.

After performing AB performance testing, we can see that the throughput has improved and the response time has decreased.

Summary #

JVM memory tuning is usually complementary to GC tuning. Based on the above tuning, we can continue to optimize the garbage collection algorithm for the young generation and heap memory. By combining the content of the previous lesson, we can complete JVM tuning together.

Although I have shared some common methods of JVM memory allocation tuning, I still recommend that you continue to use the default JVM parameters if you do not find any outstanding performance bottlenecks after performance testing. At least in most scenarios, the default configuration should already meet our requirements. But if it doesn’t, don’t panic. Combine what you have learned today and practice it. I believe you will gain new insights.

Thought Questions #

Above, we have discussed optimizing system performance by using heap memory allocation. However, in NIO socket communication, direct memory (off-heap memory) is also used to reduce memory copy and optimize socket communication. Do you know how direct memory is created and released?