26 How to Monitor and Diagnose Jvm Heap and Non Heap Memory Usage

要监控和诊断JVM堆内和堆外内存使用，我们可以使用以下方法：

监控JVM堆内内存使用 #

  • 使用JVM参数：可以使用-Xmx和-Xms参数设置堆内存的上限和初始大小。通过监控堆内存的使用情况，我们可以确定是否需要调整这些参数。
  • 使用JVM工具：JVM提供了诸如jstat、jcmd和VisualVM等工具，可以实时监控JVM的内存使用情况。这些工具可以显示堆内存的使用量、垃圾回收情况等信息，对于诊断内存问题非常有用。

监控JVM堆外内存使用 #

  • 使用JVM参数：可以使用-XX:MaxDirectMemorySize参数设置堆外内存的上限。监控堆外内存使用情况时，我们可以观察是否接近或超过了这个上限。
  • 使用系统工具：可以使用操作系统的工具，如top、ps和jcmd等命令，来监控进程的内存使用情况。通过查看进程的内存占用情况，我们可以确定是否有内存泄漏或者其他内存问题。

以上就是监控和诊断JVM堆内和堆外内存使用的方法。通过监控和诊断，我们可以及时发现和解决内存问题，提高系统的性能和稳定性。

26 How to monitor and diagnose JVM heap and non-heap memory usage #

There are many methods to understand JVM memory, and the specific capabilities vary. Here is a simple summary:

  • You can use comprehensive graphic tools such as JConsole and VisualVM (Note: Starting from Oracle JDK 9, VisualVM is no longer included in the JDK installation package). These tools are relatively intuitive to use. You can connect directly to the Java process and then grasp the memory usage in the graphical interface.

Taking JConsole as an example, its memory page can display the usage status of common heap memory and various off-heap parts.

  • You can also use command-line tools for runtime querying, such as jstat and jmap, which provide options to view data on heap, method area, etc.
  • Alternatively, you can use commands provided by jmap to generate heap dump files, and then use heap dump analysis tools such as jhat or Eclipse MAT for detailed analysis.
  • If you are using Java EE servers like Tomcat or Weblogic, these servers also provide memory management related functionalities.
  • Additionally, to some extent, outputs such as GC logs also contain rich information.

There is a relatively special part here, which is the direct memory in off-heap memory. The aforementioned tools are not suitable for it. You can use the Native Memory Tracking (NMT) feature provided by the JDK, which interprets from the perspective of memory allocation in the JVM’s native memory.

Exam Point Analysis #

Today, the selected question is about basic practices related to Java memory management. For general memory issues, mastering the typical tools and methods I provided above is sufficient. This question can also be understood as assessing two basic skills: first, whether you truly understand the internal structure of the JVM; second, when it comes to specific memory regions, what tools or features should be used for locating and what parameters can be adjusted.

The details of using tools like JConsole are not elaborated in this column. If you haven’t encountered it yet, you can refer to the official JConsole tutorial. Here, I particularly recommend Java Mission Control (JMC), which is a very powerful tool that can not only perform ordinary management and monitoring tasks using JMX, but also collect and analyze low-overhead Profiling, event, and other information about the JVM’s underlying structure in conjunction with Java Flight Recorder (JFR) technology. Oracle has already made it open source. If you are interested, you can check out the Mission Control project of OpenJDK.

Regarding memory monitoring and diagnosis, in the knowledge extension section, I will combine JVM parameters and features to try to extract a relatively clear framework from the plethora of concepts and JVM parameter options:

  • Refine the understanding of each part of the memory region, what is the structure of the heap? How to adjust it through parameters?
  • What does the off-heap memory include? What factors affect its specific size?

Knowledge Expansion #

In today’s analysis, I will combine relevant JVM parameters and tools to compare and deepen your understanding of memory areas in finer granularity.

First, what is the internal structure of the heap?

For the heap memory, in the previous lecture, I introduced the common division of young generation and old generation. With the development of JVM and the introduction of new GC methods, the internal structure of the heap can be understood from different perspectives. The following figure shows a diagram of the heap structure from a generational perspective.

As you can see, according to the usual GC generational division, the Java heap is divided into:

  1. Young generation

The young generation is the area where most objects are created and destroyed. In typical Java applications, the majority of object lifetimes are very short. The young generation is further divided into the Eden space, which is the initial allocation area for objects, and two Survivor spaces, sometimes called the “from” and “to” spaces, which are used to store objects retained from Minor GC.

  • The JVM arbitrarily selects one Survivor space as the “to” space and performs inter-area copying during the GC process, which means copying surviving objects from the Eden space and the “from” space to this “to” space. This design is mainly to prevent memory fragmentation and further clean up unused objects.
  • From the perspective of memory model rather than garbage collection, the Eden space is further divided. The Hotspot JVM has a concept called Thread Local Allocation Buffer (TLAB), and I know that all JVMs derived from OpenJDK provide TLAB design. This is a private cache area allocated by the JVM for each thread. Otherwise, when multiple threads allocate memory simultaneously, to avoid operating on the same address, locking and other mechanisms may need to be used, which will affect the allocation speed. You can refer to the diagram below. From the diagram, you can see that TLAB is still on the heap, and it is allocated within the Eden space. Its internal structure is relatively intuitive and easy to understand. The “start” and “end” represent the start address, and the “top” (pointer) indicates how much has been allocated. So when we allocate a new object, the JVM will move the “top” pointer. When the “top” and “end” meet, it means that the cache is full, and the JVM will try to allocate another block from the Eden space.

  1. Old generation

The old generation is where long-lived objects are placed. Usually, they are objects copied from the Survivor space. Of course, there are special cases. We know that ordinary objects are allocated on TLAB; if an object is relatively large, the JVM will try to allocate it directly in other positions within the Eden space; if the object is too large and cannot find a sufficiently long continuous free space in the young generation, the JVM will allocate it directly in the old generation.

  1. Permanent generation

This part represents the implementation of the method area in the early Hotspot JVM, storing Java class metadata, constant pool, and intern string cache. There is no such part after JDK 8.

So, how can we use JVM parameters to directly control the size of the heap and internal areas? Let me summarize briefly:

  • Maximum heap size
-Xmx value
  • Initial minimum heap size
-Xms value
  • Ratio of old generation size to young generation size
-XX:NewRatio=value

By default, this value is 2, which means the old generation is twice as large as the young generation. In other words, the young generation is one-third of the heap size.

  • Of course, you can also adjust the size of the young generation directly without using a ratio, by specifying the following parameter and setting a specific memory size value.
-XX:NewSize=value
  • The sizes of Eden and Survivor spaces are set proportionally. If the SurvivorRatio is 8, then the Survivor space is 1/8 the size of the Eden space, which is 1/10 of the young generation because YoungGen = Eden + 2 * Survivor. The JVM parameter format is
-XX:SurvivorRatio=value
  • TLAB can also be adjusted. The JVM implements a complex adaptive strategy. If you are interested, you can refer to this explanation.

I don’t know if you have noticed, but in the diagram of the heap structure from a generational perspective, which is the first image, I also marked the “Virtual” area. What does this area mean?

Internally in the JVM, if Xms is less than Xmx, the size of the heap does not directly expand to its upper limit. In other words, the reserved space is greater than the actually usable space. When memory needs continue to grow, the JVM gradually expands the sizes of the young generation and other areas. Therefore, the Virtual area represents the temporarily unavailable (uncommitted) space.

Secondly, after analyzing the memory spaces within the heap, let’s take a look at what the JVM’s off-heap memory actually includes.

In the memory management interfaces of JMC or JConsole, some non-heap memory is counted, but the information provided is relatively limited. The following figure shows a screenshot of the active memory pools in JMC.

Next, I will analyze the JVM with the help of the NMT feature. The detailed categorization information provided by NMT is very helpful for understanding the internal implementation of the JVM. First, let’s do some preparation work and enable NMT in summary mode:

-XX:NativeMemoryTracking=summary

To easily access and compare NMT output, choose to print NMT statistics when the application exits:

-XX:+UnlockDiagnosticVMOptions -XX:+PrintNMTStatistics

Next, execute a simple program that prints “HelloWorld” to standard output. This will generate the following output:

Now, let’s analyze the JVM native memory usage represented by NMT:

  • The first part is clearly the Java heap, which I’ve already analyzed with respect to parameter adjustments, so I won’t go into detail again.

  • The second part is the memory occupied by classes. This represents the space occupied by Java class metadata. The size can be adjusted using parameters like:

-XX:MaxMetaspaceSize=value

In this example, since there are no user libraries in HelloWorld, the memory usage is mainly from the core libraries loaded by the Bootstrap class loader. You can use the following trick to adjust the metadata space of the Bootstrap class loader for comparison and better understanding. However, this may only be meaningful when hacking the JDK:

-XX:InitialBootClassLoaderMetaspaceSize=30720
  • Next is the Thread section, which includes both Java threads such as the main thread, Cleaner threads, and native threads such as GC threads. Did you notice that even for a HelloWorld program, there are still 25 threads? It seems like a lot of waste. Imagine if we want to use Java as a Serverless runtime, where each function is very short-lived. How can we reduce the number of threads?

If you have a good understanding of the content explained in this column and have a sufficient understanding of the JVM internals, the approach becomes clear:

The default garbage collector (GC) in JDK 9 is G1, which performs well in larger heap scenarios. However, it is more complex than traditional collectors like Parallel GC or Serial GC. So either reduce the number of parallel threads used by G1 or switch to a different GC type entirely.

JIT compilation is enabled by default with TieredCompilation. Disabling it makes the JIT simpler and reduces the number of native threads accordingly.

Let’s compare the default parameters:

And now let’s replace the default GC with the Serial GC and disable TieredCompilation:

The resulting statistics show a decrease in the number of threads from 25 to 17, and a memory reduction of about one-third.

  • Next is the Code section, which obviously represents CodeCache-related memory. This is where the JIT compiler stores information about compiled hot methods, etc. The JVM provides a set of parameters to limit its initial and maximum size, for example:
-XX:InitialCodeCacheSize=value
-XX:ReservedCodeCacheSize=value

You can set these JVM parameters to further evaluate the impact of different parameters on CodeCache size.

It is evident that the CodeCache space greatly decreases by disabling the complex TieredCompilation and limiting its initial size.

  • Next is the GC section. As I mentioned before, garbage collectors like G1 have complex and large data structures and mechanisms. For example, the Remembered Set usually occupies around 20% to 30% of the heap space. What if we explicitly switch to the simpler Serial GC?

Use the following command:

-XX:+UseSerialGC

As you can see, not only does the total number of threads decrease significantly (25 to 13), but the memory overhead of the GC mechanism itself is also greatly reduced. I know for a fact that the Java runtime in AWS Lambda uses the Serial GC, which significantly reduces the startup and runtime overhead of individual functions.

  • The Compiler section represents the overhead of the JIT, and obviously, disabling TieredCompilation reduces memory usage.

  • Other sections have very low proportions and usually do not cause memory usage issues. Please refer to the official documentation. The only exception is the Internal (or Other in JDK 11 and later) section, which includes memory allocated for Direct Buffers. This is a sensitive part of off-heap memory, and many off-heap memory OOM (Out of Memory) errors occur in this area. Please refer to Step 12 of this column for the handling steps. In principle, frequent creation or destruction of Direct Buffers is not recommended. If you suspect issues with direct memory areas, you can usually investigate potential problems using techniques such as instrumenting constructors.

This concludes the introduction to the JVM internal structure. The main goal is to deepen your understanding. Many aspects are only relevant when customizing or tuning the JVM runtime. With the rise of microservices and serverless technologies, there is a demand for customizing the JDK for new workloads.

Today, I systematically analyzed the JVM heap and off-heap memory structures with JVM parameters and features. I believe you now have a deep understanding of the JVM memory structure, and your thought process will be clearer when customizing the Java runtime or dealing with OOM issues. JVM problems vary greatly, but if you can quickly narrow down the problem, you will have a good idea of where the issue might be. For example, if you can pinpoint a potential heap memory leak, there are often clear approaches and tools to solve it.

Practice Exercise #

Have you been able to grasp the topic we discussed today? Today’s thought-provoking question is: What techniques can be used to monitor Java memory usage programmatically rather than using tools?

Please write your thoughts on this question in the comments section. I will select the most carefully considered comments and reward you with a learning voucher. Feel free to join me in the discussion.

Are your friends also preparing for interviews? You can “invite friends to read” and share today’s topic with them. Perhaps you can help them out.