24 Memory Analysis and Related Tools Part 2- Common Issue Analysis #
The memory of a Java program can be divided into several parts: heap space, non-heap, stack, etc., as shown in the following diagram:
The most common java.lang.OutOfMemoryError can be categorized into the following types.
OutOfMemoryError: Java heap space #
JVM limits the maximum memory usage of a Java program, which is determined by the JVM’s startup parameters.
Among them, the maximum value of heap memory is specified by the JVM startup parameter -Xmx. If it is not explicitly specified, the default value is calculated based on the platform type (OS version + JVM version) and the size of physical memory.
If there is not enough space in the heap memory to accommodate a newly created object, it will result in an “java.lang.OutOfMemoryError: Java heap space” error. This error will be thrown as long as the heap memory usage reaches the maximum memory limit, regardless of whether there is any free physical memory on the machine.
Cause Analysis #
The cause of the “java.lang.OutOfMemoryError: Java heap space” error is often similar to trying to fit an XXL-sized object into an S-sized Java heap space. Once the cause is clear, the problem is easily resolved: just increase the size of the heap memory, and the program will run normally. In addition, there are some more complex situations mainly caused by code issues:
- Exceeding the expected access/data volume: When designing an application system, there is usually a “capacity” defined, which involves deploying a certain number of machines to handle a certain amount of data/business. If the access volume suddenly surges and exceeds the expected threshold, it will create a graph similar to a needle shape on a time coordinate system. In such cases, during the time period where the peak occurs, the program may freeze and trigger a “java.lang.OutOfMemoryError: Java heap space” error.
- Memory leak: This is also a common scenario. Due to certain hidden errors in the code, the system’s memory usage continues to increase. If there is a memory leak in a certain method/code segment, every time it is executed, more garbage objects will occupy more memory. As time goes on, the leaked objects deplete all the memory in the heap, resulting in the eruption of the “java.lang.OutOfMemoryError: Java heap space” error.
A Very Simple Example #
The following code is very simple. The program attempts to allocate an int array with a capacity of 16MB. If the startup parameter is specified as -Xmx16m, then an “java.lang.OutOfMemoryError: Java heap space” error will occur. However, if the parameter is slightly modified to -Xmx20m, the error will not occur anymore.
public class OOM {
static final int SIZE = 16 * 1024 * 1024;
public static void main(String[] a) {
int[] i = new int[SIZE];
}
}
Solution #
If the maximum memory set does not meet the requirements for normal program execution, simply increase the heap memory. You can refer to the following text for configuring parameters. However, in many cases, increasing the heap memory space cannot solve the problem. For example, if there is a memory leak, increasing the heap memory will only delay the triggering of the “java.lang.OutOfMemoryError: Java heap space” error.
Of course, increasing the heap memory may increase the GC pause time, thus affecting the program’s throughput or latency.
If you want to solve the problem fundamentally, you need to investigate the code that allocates memory. In simple terms, you need to figure out the following questions:
- Which type of object occupies the most memory?
- Where are these objects allocated in the code?
To understand this, it may take a lot of time for analysis. Here is a rough process:
- Obtain the permission to perform a heap dump on the production server. A dump is a snapshot of the heap memory that can be used for memory analysis. These snapshots may contain confidential information such as passwords and credit card numbers, so it is not easy to obtain heap dumps from a production environment due to security restrictions. In such cases, you need to ensure that the sensitive information is not leaked by using some security policies (such as using desensitization mechanisms).
- Perform a heap dump at the appropriate time. Generally, memory analysis requires comparing multiple heap dump files. If the timing is not right, then it might be an “invalid” snapshot. Also, each heap dump freezes the JVM, so in a production environment, you cannot freely perform too many dump operations, otherwise the system may slow down or hang, causing more trouble.
- Load the dump file on another machine. Typically, if the problematic JVM memory is 8GB, then the machine used to analyze the heap dump needs to have more than 8GB of memory. Open a dump analysis software (we recommend Eclipse MAT, but you can also use other tools).
- Identify the GC roots that occupy the most memory in the snapshot. This may be a bit difficult for beginners, but it will deepen your understanding of the heap memory structure and other mechanisms.
- Next, identify the code that may allocate a large number of objects. If you are very familiar with the entire system, you may be able to locate it quickly.
Generally, with this information, we can help pinpoint the root cause of the problem and take targeted measures, such as streamlining data structures/models appropriately to only occupy necessary memory and solve the problem.
Of course, based on the results of memory analysis, if it is found that the memory occupied by objects is reasonable and there is no need to modify the source code, then modify the JVM startup parameters and increase the heap memory. This is a simple and effective way to make the system work happily and run more smoothly.
OutOfMemoryError: GC overhead limit exceeded #
The Java runtime environment has a built-in garbage collection (GC) module. Many previous generations of programming languages did not have automatic memory management, and programmers needed to manually allocate and release memory to reuse heap memory. In a Java program, you only need to focus on memory allocation. If a piece of memory is no longer in use, the garbage collection module will automatically clean it up. The detailed principles of GC can be found in the series of articles on GC performance optimization. Generally, the built-in garbage collection algorithm of JVM can handle the vast majority of business scenarios.
The reason for the “java.lang.OutOfMemoryError: GC overhead limit exceeded” error is that the program has basically exhausted all available memory and the GC cannot clean it.
Analysis of the Causes #
When JVM throws the “java.lang.OutOfMemoryError: GC overhead limit exceeded” error, it is sending a signal that the proportion of time spent on garbage collection is too large compared to the effective computation. By default, if the time spent by the GC exceeds 98% and the amount of memory reclaimed is less than 2%, the JVM will throw this error. In other words, the system cannot work properly as most resources are used for GC, but the GC has little effect. At this point, the system is like being in the late stage of cancer, where the cancer cells occupy all the nutrients of the body, and very little is actually available for the body to use. Even if all the nutrients are used to kill the cancer cells, it is too late because the killing effect is very poor and the replication speed of the cancer cells is much faster.
Note that the “java.lang.OutOfMemoryError: GC overhead limit exceeded” error is only thrown in extreme cases where multiple consecutive GCs reclaim less than 2% of memory. What would happen if the GC overhead limit error was not thrown? In that case, the little memory cleaned up by GC would quickly be filled again, forcing GC to execute once more. This creates a vicious cycle where the CPU usage remains at 100% while GC achieves no results. Users will see the system freeze - an operation that used to take only a few milliseconds now takes several minutes or even hours to complete.
This is also a good example of the fail-fast principle.
Example #
Let’s simulate the phenomenon. The following code adds data to a Map in an infinite loop, which will result in a “GC overhead limit exceeded” error:
package com.cncounter.rtime;
import java.util.Map;
import java.util.Random;
public class TestWrapper {
public static void main(String args[]) throws Exception {
Map map = System.getProperties();
Random r = new Random();
while (true) {
map.put(r.nextInt(), "value");
}
}
}
Configure the JVM parameter -Xmx12m, and the error message generated after execution is as follows:
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Hashtable.addEntry(Hashtable.java:435)
at java.util.Hashtable.put(Hashtable.java:476)
at com.cncounter.rtime.TestWrapper.main(TestWrapper.java:11)
The error message you encounter may not necessarily be the same. Indeed, the JVM parameters we executed are:
java -Xmx12m -XX:+UseParallelGC TestWrapper
We quickly see the “java.lang.OutOfMemoryError: GC overhead limit exceeded” error prompt message. However, this example is somewhat tricky because different heap memory sizes and different GC algorithms used will result in different error messages. For example, when the Java heap memory is set to 10M (too small to exhaust before the system can reclaim it):
java -Xmx10m -XX:+UseParallelGC TestWrapper
In DEBUG mode, the error message is as follows:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Hashtable.rehash(Hashtable.java:401)
at java.util.Hashtable.addEntry(Hashtable.java:425)
at java.util.Hashtable.put(Hashtable.java:476)
at com.cncounter.rtime.TestWrapper.main(TestWrapper.java:11)
Readers should try modifying the parameters and execute to see the specific results. The error prompts and stack trace information may vary.
Here, when the Map executes the rehash method, it throws a “java.lang.OutOfMemoryError: Java heap space” error message. If you use other garbage collection algorithms, such as -XX:+UseConcMarkSweepGC or -XX:+UseG1GC, the error will be caught by the default exception handler, but without stack trace information, because when creating the Exception, it is not possible to populate the stack trace information.
For example, configure:
-Xmx12m -XX:+UseG1GC
Executed on a Win7x64, Java 8 environment, the generated error message may be:
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
It is recommended that readers modify the memory configuration and garbage collector for testing.
These real cases illustrate that when resources are limited, it is impossible to accurately predict the specific reason for the program’s termination. Therefore, in the face of such errors, it is not advisable to bind a specific error handling order.
Solution #
There is a solution to simply handle the situation without throwing the “java.lang.OutOfMemoryError: GC overhead limit exceeded” error message, by adding the following startup parameter:
// Not recommended
-XX:-UseGCOverheadLimit
We strongly advise against specifying this option because it does not truly solve the problem, but only postpones the occurrence of the “out of memory” error. Eventually, other actions will still need to be taken. Specifying this option will mask the original “java.lang.OutOfMemoryError: GC overhead limit exceeded” error and instead produce the more common “java.lang.OutOfMemoryError: Java heap space” error message.
Note: Sometimes the reason for triggering the GC overhead limit error is insufficient heap memory allocated to the JVM. In this case, simply increasing the heap memory size should suffice.
In most cases, increasing the heap memory will not solve the problem. For example, if there is a memory leak in the program, increasing the heap memory will only postpone the occurrence of the “java.lang.OutOfMemoryError: Java heap space” error.
It is worth reiterating that increasing the heap memory may also increase the time for GC pauses, thus affecting the program’s throughput or latency.
If you want to solve the problem fundamentally, you need to investigate the memory allocation-related code and use tools for further analysis and diagnosis. Refer to the previous section for specific steps.
OutOfMemoryError: PermGen space #
Note: PermGen (Permanent Generation) is a concept in JDK 1.7 and earlier versions. With the development of Java, starting from JDK 8, it has been replaced by a less restricted Metaspace. For more details, please refer to the next article: [OutOfMemoryError Series (4): Metaspace]
The “java.lang.OutOfMemoryError: PermGen space” error message indicates that the Permanent Generation memory area is full.
Analysis of the cause #
First, let’s see what PermGen is used for.
In JDK 1.7 and earlier versions, PermGen was mainly used to store class definitions that were loaded/cached into memory, including the class name, fields, methods, bytecode, constant pool information, classes associated with object arrays/type arrays, and optimized class information by the JIT compiler.
It is easy to see that the usage of PermGen is related to the number/size of classes loaded into memory by the JVM. The main reason for “java.lang.OutOfMemoryError: PermGen space” is that the number or size of classes loaded into memory exceeds the size of the PermGen area.
Example #
The following code demonstrates this situation:
import javassist.ClassPool;
public class MicroGenerator {
public static void main(String[] args) throws Exception {
for (int i = 0; i < 100_000_000; i++) {
generate("jvm.demo.Generated" + i);
}
}
public static Class generate(String name) throws Exception {
ClassPool pool = ClassPool.getDefault();
return pool.makeClass(name).toClass();
}
}
In this code, many classes are dynamically generated within a for loop. (As you can see, generating classes using the javassist utility class is very simple.)
When executing this code, many new classes will be generated and loaded into memory. As more and more classes are generated, the PermGen space will be filled, and eventually an “java.lang.OutOfMemoryError: PermGen space” error will be thrown (although it is also possible to throw other types of OutOfMemoryError).
To see the effect quickly, you can add appropriate JVM startup parameters, such as -Xmx200M -XX:MaxPermSize=16M.
OutOfMemoryError caused by redeployment #
The following scenario is more common: when redeploying a web application to a container such as Tomcat, it is likely to cause a “java.lang.OutOfMemoryError: PermGen space” error.
Ideally, when redeploying, containers like Tomcat should use a new classloader to load new classes and allow the garbage collector to clean up the previous classloader (along with the loaded classes).
However, the reality may not be so optimistic. Many third-party libraries, as well as certain restricted shared resources such as threads, JDBC drivers, and file system handles, may prevent the previous classloader from being completely uninstalled.
Therefore, when redeploying, the previous classes will still remain in the PermGen space, generating tens or even hundreds of megabytes of garbage with each redeployment, just like eczema in memory.
For example, suppose an application loads a JDBC driver and connects to a database during startup. According to the JDBC specification, the driver will register itself with java.sql.DriverManager by adding an instance to a static field in DriverManager.
When the application is undeployed from the container, java.sql.DriverManager still holds the JDBC instance (Tomcat often issues warnings), and the JDBC driver instance holds a java.lang.Classloader instance. As a result, the garbage collector cannot reclaim the corresponding memory space.
And the java.lang.ClassLoader instance holds all the classes it loads, typically tens or hundreds of megabytes of memory. As a result, redeployment will consume another PermGen space of nearly the same size, and after multiple redeployments, a “java.lang.OutOfMemoryError: PermGen space” error will occur. In the log file, you should see relevant error messages.
Solutions #
Now that we understand the problem, we can consider the corresponding solutions.
1. Solution for OutOfMemoryError during program startup
If OutOfMemoryError occurs due to the PermGen space being exhausted during program startup, it can be easily resolved. Increase the size of PermGen to allow more memory for loading classes. Modify the -XX:MaxPermSize startup parameter, for example:
java -XX:MaxPermSize=512m com.yourcompany.YourClass
The above configuration allows the JVM to use a maximum PermGen space of 512MB, and if it is still not enough, it will throw an OutOfMemoryError.
2. Solution for OutOfMemoryError during redeployment
We can perform heap dump analysis—execute a heap dump after redeployment, similar to the following:
jmap -dump:format=b,file=dump.hprof <process-id>
Then load the dumped file with a heap dump analyzer (such as the powerful Eclipse MAT) to find duplicate classes, especially classes corresponding to class loaders. You may need to compare all class loaders to find the one currently in use.
For unused class loaders (inactive class loaders), you need to first determine the shortest path GC root that is preventing it from being garbage collected. This is the only way to find the root cause of the problem. If it is caused by a third-party library, you can search Google/StackOverflow for solutions. If it is caused by your own code, you need to modify the code to remove the relevant references at the appropriate time.
3. Solving OutOfMemoryError at Runtime
If an OutOfMemoryError occurs during runtime, you first need to confirm whether GC can unload classes from PermGen.
The official JVM is quite conservative in this regard (it keeps loaded classes in memory even if they are no longer used).
However, modern applications generally dynamically create a large number of classes during runtime, and the lifetimes of these classes are usually short. Older versions of the JVM cannot handle this problem well. Therefore, we need to allow the JVM to unload classes, for example, using the following startup parameter:
-XX:+CMSClassUnloadingEnabled
By default, the value of CMSClassUnloadingEnabled is false, so it needs to be explicitly specified. After that, GC will clean up PermGen and unload unused classes. Of course, this option only takes effect when UseConcMarkSweepGC is set. If ParallelGC or Serial GC is used, it needs to be switched to CMS.
If it is determined that classes can be unloaded and there is still an OutOfMemoryError, then heap dump analysis similar to the one used in the previous section needs to be performed.
Further reading: “Goodbye to OOM: PermGen”
OutOfMemoryError: Metaspace #
“java.lang.OutOfMemoryError: Metaspace” is an error message when the Metaspace is full.
Root Cause Analysis #
Starting from Java 8, there have been significant changes in the memory structure, and the JVM no longer uses PermGen. Instead, a new space called Metaspace is introduced. This change is based on multiple considerations, including:
- It is difficult to predict the exact size of the PermGen space. Setting it too small will result in a “java.lang.OutOfMemoryError: Permgen size” error, while setting it too large will cause waste.
- To improve GC performance and make the concurrent phase of the garbage collection process no longer pause.
- To optimize concurrent class unloading for G1 garbage collector and ensure reliable class unloading.
In Java 8, all content previously in PermGen is moved to the Metaspace, such as class names, fields, methods, bytecode, constant pools, and JIT-optimized code.
Therefore, Metaspace is basically equivalent to PermGen. Similarly, the main cause of the “java.lang.OutOfMemoryError: Metaspace” error is that too many classes or classes with large sizes are loaded into memory.
Example #
Similar to the PermGen in the previous section, the usage of Metaspace is closely related to the number of classes loaded by the JVM. Here is a simple example:
public class Metaspace {
static javassist.ClassPool cp = javassist.ClassPool.getDefault();
public static void main(String[] args) throws Exception{
for (int i = 0; ; i++) {
Class c = cp.makeClass("jvm.demo.Generated" + i).toClass();
}
}
}
The code is basically the same as the previous section and will not be described again.
When running this code, as more and more classes are generated, the Metaspace will eventually be filled up, resulting in a “java.lang.OutOfMemoryError: Metaspace” error. According to our tests, in the Java 8 environment on Mac OS X, if the startup parameter -XX:MaxMetaspaceSize=64m is set, the JVM will crash after loading approximately 70,000 classes.
Solution #
If you encounter an OutOfMemoryError related to Metaspace, the first solution is to increase the size of Metaspace (similar to PermGen), for example:
-XX:MaxMetaspaceSize=512m
Another seemingly simple solution is to remove the size limit on Metaspace.
However, it is important to note that removing the size limit of Metaspace may cause swapping between physical memory and virtual memory if there is not enough physical memory. This can severely degrade system performance and may also cause issues with native memory allocation.
Note: In modern application clusters, it is preferable to let application nodes fail fast rather than slow and steadily.
If you don’t want to receive alarms, you can hide the “java.lang.OutOfMemoryError: Metaspace” error message like an ostrich. However, this will bring more problems, so please refer to the previous sections to find a solution carefully.
OutOfMemoryError: Unable to create new native thread #
Java programs are fundamentally multithreaded, which means they can execute multiple tasks simultaneously. Similar to when you can drag and drop content in a window while playing a video without pausing the video playback, even if the physical machine has only one CPU.
Threads can be thought of as workers who do the actual work. If there is only one worker, then only one task can be executed at a time. But if there are many workers, then multiple tasks can be executed simultaneously.
Similar to the real world, threads in the JVM also need memory space to execute their tasks. If the number of threads is too large, it will introduce new problems:
The java.lang.OutOfMemoryError: Unable to create new native thread error is an exception message that indicates the number of threads created by the program has reached the maximum limit.
Cause Analysis #
When the JVM requests to create a new Java thread from the operating system, it may encounter the java.lang.OutOfMemoryError: Unable to create new native thread error. If the underlying operating system fails to create a new native thread, the JVM will throw the corresponding OutOfMemoryError.
In general, the scenarios that cause the java.lang.OutOfMemoryError: Unable to create new native thread error can go through the following stages:
- The Java program requests the JVM to create a new Java thread.
- The JVM native code handles the request and tries to create an operating system-level native thread.
- The operating system tries to create a new native thread, which requires allocating some memory for that thread.
- If the operating system’s memory is exhausted or limited by the address space of a 32-bit process (about 2-4GB), the OS will refuse the local memory allocation.
- The JVM throws the
java.lang.OutOfMemoryError: Unable to create new native threaderror.
Example #
The following code creates and starts many new threads in an infinite loop. After executing the code, the operating system limit is quickly reached, resulting in the java.lang.OutOfMemoryError: unable to create new native thread error.
package demo.jvm0205;
import java.util.concurrent.TimeUnit;
/**
* Demonstrates: java.lang.OutOfMemoryError: unable to create new native thread
*/
public class UnableCreateNativeThread implements Runnable {
public static void main(String[] args) {
UnableCreateNativeThread task = new UnableCreateNativeThread();
int i = 0;
while (true){
System.out.println("Trying to create: " + (i++));
// Continuously create threads
new Thread(task).start();
}
}
@Override
public void run() {
try {
TimeUnit.SECONDS.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
When running the code on a Mac OS X machine with 16GB of memory using IntelliJ IDEA, the result is as follows:
Trying to create: 0
......
Trying to create: 4069
Trying to create: 4070
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at demo.jvm0205.UnableCreateNativeThread.main(UnableCreateNativeThread.java:16)
After that, the machine’s operating system crashes and restarts.
We can also modify the code to make the JVM exit directly if the exception is caught:
// Continuously create threads
try {
new Thread(task).start();
} catch (Throwable e){
System.exit(0);
}
Executing again, only the application will report an error and exit. It is estimated that IDEA has registered some hooks for memory overflow errors, which temporarily prevents the JVM from exiting, leading to system crashes.
The number of native threads is determined by the specific environment, such as on Windows, Linux, and Mac OS X systems:
- 64-bit Mac OS X 10.9, Java 1.7.0_45 - JVM crashes after creating thread #2031.
- 64-bit Ubuntu Linux, Java 1.7.0_45 - JVM crashes after creating thread #31893.
- 64-bit Windows 7, Java 1.7.0_45 - This error message does not seem to appear because the operating system uses a different thread model. After creating thread #250,000, the Java process still exists, but the virtual memory (swap file) usage reaches 10GB, causing the system to run extremely slowly and almost freeze.
So if you want to know where the system’s limit is, you just need a small test case to find the number of threads created when “java.lang.OutOfMemoryError: Unable to create new native thread” is triggered.
Solution #
Sometimes you can modify the system restrictions to avoid the “Unable to create new native thread” problem. For example, on Linux, increase the maximum number of available file descriptors (everything is a file descriptor/FD on Linux):
[root@dev ~]# ulimit -a
core file size (blocks, -c) 0
...... omitted content ......
max user processes (-u) 1800
In most cases, triggering OutOfMemoryError when creating native threads indicates the existence of bugs in the programming. For example, if a program creates thousands of threads, there may be a big problem - there are few programs that can handle tens of thousands of threads (the number of CPUs is limited, and it is impossible for too many threads to simultaneously obtain control of the CPU to run).
One solution is to perform a thread dump to analyze the specific situation, which we will discuss in later sections.
OutOfMemoryError: Out of swap space #
JVM startup parameters specify the maximum memory limit, such as -Xmx and other related startup parameters. If the total amount of memory used by the JVM exceeds the available physical memory, the operating system will use virtual memory (generally based on disk files).
The error message “java.lang.OutOfMemoryError: Out of swap space” indicates that there is insufficient swap space (swap space/virtual memory), resulting in memory allocation failure due to insufficient physical memory and swap space.
Cause Analysis #
If the native heap memory is exhausted, the JVM will throw a “java.lang.OutOfmemoryError: Out of swap space” error message, indicating that the operation of requesting memory allocation has failed.
This error occurs only when the Java process uses virtual memory. This is a difficult scenario for garbage collection in Java. Even though modern garbage collection algorithms are advanced, the system delays caused by virtual memory swapping will greatly increase the pause time for garbage collection, making it difficult to tolerate.
Usually, “java.lang.OutOfMemoryError: Out of swap space” problems are caused by reasons at the operating system level, such as:
- The swap space of the operating system is too small.
- Some process on the machine has consumed all memory resources.
Of course, it could also be caused by native memory leaks in the application, such as a program/library continuously allocating native memory without releasing it.
Solution #
There are multiple solutions to this problem.
The first and simplest method is to increase the size of the swap space. The methods for setting the swap space vary for different operating systems. For example, on Linux, you can use the following commands to set the swap space:
swapoff -a
dd if=/dev/zero of=swapfile bs=1024 count=655360
mkswap swapfile
swapon swapfile
Here, a swap file with a size of 640MB is created and enabled.
Because the garbage collector needs to clean up the entire memory space, virtual memory is unbearable for Java GC. When there is memory swapping, the pause time for garbage collection will increase by hundreds of times. Therefore, it is best to avoid increasing virtual memory or even using virtual memory (after all, the access speed of memory differs from that of the disk by several orders of magnitude).
OutOfMemoryError: Requested array size exceeds VM limit #
The Java platform limits the maximum length of arrays. The specific limits may vary for different versions, but they are all within the range of 1 to 2.1 billion. (Think about why it is 2.1 billion?)
If the program throws a “java.lang.OutOfMemoryError: Requested array size exceeds VM limit” error, it means that the program is trying to create an array with a length exceeding the limit.
Cause Analysis #
This error is an check performed by the JVM before actually allocating memory for the array: whether the data structure to be allocated can be addressed on the platform. Of course, this error is much rarer than you think.
This error is rarely seen because Java uses the int type as the index for arrays. In Java, the maximum value of the int type is 2^31-1 = 2147483647. Most platform limits are approximately equal to this value. For example, on a 64-bit MB Pro, Java 1.7 platform can allocate arrays with a length of 2147483645 (or Integer.MAX_VALUE-2) and shorter.
Once it exceeds a little more length and becomes Integer.MAX_VALUE-1, the well-known OutOfMemoryError will be thrown:
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
On some platforms, this maximum limit can be even smaller. For example, on 32-bit Linux with OpenJDK 6, an array length of about 1.1 billion (approximately 2^30) will throw a "java.lang.OutOfMemoryError: Requested array size exceeds VM limit" error.
To find out the specific limit value, you can execute a small test case. The example below demonstrates this.
#### Example
The following code demonstrates the "java.lang.OutOfMemoryError: Requested array size exceeds VM limit" error:
for (int i = 3; i >= 0; i–) { try { int[] arr = new int[Integer.MAX_VALUE-i]; System.out.format(“Successfully initialized an array with %,d elements.\n”, Integer.MAX_VALUE-i); } catch (Throwable t) { t.printStackTrace(); } }
In this code, a for loop iterates 4 times, initializing an int array each time with a length ranging from Integer.MAX_VALUE-3 to Integer.MAX_VALUE.
On Hotspot 7 platform on 64-bit Mac OS X, executing this code will result in something similar to the following:
java.lang.OutOfMemoryError: Java heap space at jvm.demo.ArraySize.main(ArraySize.java:8) java.lang.OutOfMemoryError: Java heap space at jvm.demo.ArraySize.main(ArraySize.java:8) java.lang.OutOfMemoryError: Requested array size exceeds VM limit at jvm.demo.ArraySize.main(ArraySize.java:8) java.lang.OutOfMemoryError: Requested array size exceeds VM limit at jvm.demo.ArraySize.main(ArraySize.java:8)
Please note that before the last two iterations, which throw the "java.lang.OutOfMemoryError: Requested array size exceeds VM limit" error, two "java.lang.OutOfMemoryError: Java heap space" errors are thrown.
This is because an array of 2^31-1 integers exceeds the default 8GB heap memory of the JVM.
This example also demonstrates the rarity of this error. To obtain the JVM's restriction on array size, an array with a length close to Integer.MAX_INT must be allocated. When running this example on 64-bit Mac OS X with Hotspot 7, only two lengths will throw this error: Integer.MAX_INT-1 and Integer.MAX_INT.
#### Solution
This error is uncommon. The "java.lang.OutOfMemoryError: Requested array size exceeds VM limit" error may occur due to the following reasons:
- The array is too large, and its final length exceeds the platform's limit but is still less than Integer.MAX_INT.
- A large array exceeding 2^31-1 is intentionally allocated to test system limitations.
In the first case, you need to check your business code and determine if such a large array is really necessary. If possible, reducing the array length would be beneficial.
In the second case, remember that Java arrays use int values as indices. Therefore, the number of array elements cannot exceed 2^31-1. In fact, the code will produce a compilation error at the stage with a message like "error: integer number too large". If you really need to handle large datasets, consider adjusting your solution. For example, you could divide the data into smaller chunks and load them in batches or abandon the use of standard libraries and handle the data structure yourself. You can use the `sun.misc.Unsafe` class to directly allocate memory, similar to C language behavior.
### OutOfMemoryError: Kill process or sacrifice child
In summary, this error can be described as:
> Killing processes when necessary...
To understand this error, let's review some basic knowledge related to operating systems.
As we know, an operating system is built on top of processes. Processes are scheduled and maintained by kernel jobs, one of which is known as the "Out of memory killer (OOM killer)" and is related to the OutOfMemoryError discussed in this section.
When available memory is extremely low, the OOM killer may kill certain processes. It activates when certain conditions are met and selects a process with the lowest score to kill using a heuristic scoring algorithm.
Therefore, "OutOfMemoryError: Kill process or sacrifice child" is different from the previously discussed OutOfMemoryError because it is not triggered by the JVM or its agent; instead, it is a built-in safety measure of the system's kernel.
If available memory (including swap) becomes insufficient, it can affect system stability. In such cases, the OOM killer will try to identify and kill rogue processes to free up memory, triggering the "OutOfMemoryError: Kill process or sacrifice child" error.
#### Cause Analysis
By default, Linux kernels allow processes to request more memory than the system has available. This is because in most cases, many processes request more memory than they actually need.
Here's a simple analogy: Internet service providers can oversell their bandwidth, selling more than 100 subscriptions for 100Mbps even though their total bandwidth is only 10Gbps. This is possible because most of the time, the bandwidth usage of individual subscribers is staggered, and it is unlikely that all subscribers will fully utilize the promised bandwidth at the same time.
Overselling creates a problem. If certain programs consume a large amount of system memory, the available memory becomes very small, and there are no memory pages available to allocate to processes that genuinely need it. To prevent this situation, the system will automatically activate the OOM killer, which identifies rogue processes and terminates them.
For more details on performance optimization of "Out of memory killer", please refer to the [RedHat official documentation](https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-captun.html).
Now that we know why this problem occurs, why does the "killer" trigger an alert message at 5 o'clock in the morning? Typically, the triggering reason is related to the OS configuration. For example, the value of the configuration file `/proc/sys/vm/overcommit_memory` specifies whether all `malloc()` calls are allowed to succeed.
> Please note that the path to this configuration file may vary among different operating systems.
#### **Example**
Compile and execute the following example code on Linux (such as the latest stable version of Ubuntu):
```java
package jvm.demo;
public class OOM {
public static void main(String[] args){
java.util.List<int[]> l = new java.util.ArrayList();
for (int i = 10000; i < 100000; i++) {
try {
l.add(new int[100_000_000]);
} catch (Throwable t) {
t.printStackTrace();
}
}
}
}
You will see an error in the system log (such as the /var/log/kern.log file), similar to this:
Jun 4 07:41:59 jvm kernel:
[70667120.897649]
Out of memory: Kill process 29957 (java) score 366 or sacrifice child
Jun 4 07:41:59 jvm kernel:
[70667120.897701]
Killed process 29957 (java) total-vm:2532680kB, anon-rss:1416508kB, file-rss:0kB
Note: The above example may require adjusting the size of swap and setting the maximum heap memory, for example, the heap memory configuration is
-Xmx2g, and the swap is configured as follows:
swapoff -a
dd if=/dev/zero of=swapfile bs=1024 count=655360
mkswap swapfile
swapon swapfile
Solution #
There are multiple ways to solve this problem. The simplest way is to migrate the system to an instance with more memory.
In addition, you can also optimize the OOM killer or balance the load (horizontal scaling, clustering), or reduce the memory requirements of the application.
It is not recommended to increase swap space/virtual memory.
Consider this: Java includes automatic garbage collection, and increasing swap memory comes at a high cost. Modern GC algorithms perform extremely fast when dealing with physical memory (and memory prices are getting cheaper).
But when it comes to swap memory, it’s not efficient. Swap memory can lead to several orders of magnitude increase in GC pause times. Therefore, before adopting this solution, consider whether it is really necessary.
Other Out of Memory Errors #
In fact, there are various other types of OutOfMemoryError.
“java.lang.OutOfMemoryError: reason stack_trace_with_native_method” is generally caused by:
- Memory allocation failure in native threads.
- The top frame in the call stack belongs to a native method when printed.
Here are a few more examples:
- Article on HelloJava WeChat Subscription Account - java.lang.OutOfMemoryError:Map failed
- Article on hellojavacases WeChat Subscription Account - java.lang.OutOfMemoryError:Direct Buffer Memory
- Oracle official documentation - Understand the OutOfMemoryError Exception
These errors are rare, and if encountered, you need to diagnose them on a case-by-case basis using the debug tools provided by the operating system.
Summary #
In this section, we reviewed various types of OutOfMemoryError, which can cause our systems to crash and become unresponsive under various conditions. By analyzing these examples and solutions, we can confidently use systematic methods to deal with these issues and make our systems more stable and efficient.