30 What Are the New Issues When Java Programs Run in Container Environments Like Docker

30 What are the new issues when Java programs run in container environments like Docker #

Today, Docker and other container technologies are no longer new concepts. They are becoming an integral part of daily development and deployment environments. Whether Java can seamlessly run in container environments and meet the requirements of new software architectures and scenarios such as microservices and serverless computing will also have an impact on future technology stack choices to some extent. Of course, the support for Docker and other container environments in Java is also continuously improving. Naturally, the practical aspects of running Java in container scenarios are gradually being discussed in interviews. I hope that through today’s lecture in this column, I can help you feel confident and prepared.

The question I want to ask you today is, What are the new issues when running Java programs in Docker and other container environments?

Typical Answer #

For Java, Docker is still a relatively new environment. For example, its memory, CPU, and other resource limits are implemented through CGroup (Control Group). Early versions of JDK (before 8u131) cannot recognize these limitations, which can cause some basic problems:

If the appropriate JVM heap and metaspace, direct memory, and other parameters are not configured, Java may attempt to use more memory than the container limit, resulting in the container being OOM killed or experiencing its own OOM.
Incorrect judgment of available CPU resources. For example, if Docker limits the number of CPUs, JVM may set an inappropriate number of parallel GC threads.

From the perspective of application packaging and deployment, JDK itself is relatively large, so the generated image becomes even more bloated. When we have a large number of images, the overhead of storing the images becomes more obvious.

If we consider new architectures and scenarios such as microservices and serverless, Java itself has certain limitations in terms of size, memory usage, and startup speed, because most of Java’s early optimizations were targeted at long-running large-scale server-side applications.

Analysis of the Test Point #

Today’s question is a question targeting a specific scenario and knowledge point. The answer I provided briefly summarizes some of the issues found in current industry practices.

If I were the interviewer, for this type of question, if you really don’t have much experience using Java in a Docker environment, it is acceptable to say that you don’t know. After all, no one can master all knowledge points.

However, we should be clear that experienced interviewers generally do not use purely esoteric knowledge points as the purpose of the interview. Rather, they are more interested in examining the thought process for problem-solving and the methods used to solve problems. Therefore, if you have a foundation, you can demonstrate your ability to provide a systematic analysis of new problems and scenarios from the perspectives of operating systems, container principles, JVM internal mechanisms, software development practices, etc. After all, change is always the theme of the world, and the ability to identify commonalities and key points in new changes is an essential skill for outstanding engineers.

Today, I will expand on the following aspects:

The interviewer may further ask, have you ever thought about why container environments like Docker “bully” Java? From the perspective of the JVM internal mechanism, where does the problem arise?
I notice that there is an argument saying “no one uses Java in a container environment”. Without debating the correctness of this view, I will start from engineering practice, sort out the reasons for the problem, discuss related solutions, and explore the best practices in new scenarios.

Knowledge Expansion #

Firstly, let’s clarify the limitations of Java in container environments and find out what makes Docker special.

Although Docker and virtual machines may seem very similar, for example, they both have their own shell and can install software packages independently, and their runtimes do not interfere with each other. However, upon closer analysis, you will find that Docker is not a complete virtualization technology, but rather a lightweight isolation technology.

The above diagram shows the difference between Docker and virtual machines. From a technical standpoint, Docker provides separate namespaces for each container based on namespaces, achieving isolation in areas such as networking, PID, users, IPC communication, and file system mounts. Resource management such as CPU, memory, and disk IO are managed through CGroups. If you want to learn more about Docker, please refer to the relevant technical documentation.

Docker only implements limited isolation and virtualization on top of Linux-like kernels. It does not run a new operating system independently like traditional virtualization software. If it is a virtualized operating system, whether it is Java or other programs, as long as they call the same system API, they can transparently obtain the required information without significant compatibility changes.

While containers eliminate the overhead of virtual operating systems and achieve lightweight goals, they also bring additional complexity. Containers are not transparent to applications, and users need to understand Docker’s new behavior. Therefore, as experts have said, “Fortunately, Docker does not completely hide the underlying information, but unfortunately Docker does not hide the underlying information!”

For the Java platform, these exposed underlying information can lead to unexpected difficulties, mainly in the following aspects:

First, the management of computing resources in container environments is completely different. CGroups, as a relatively new technology, older versions of Java obviously cannot naturally understand the corresponding resource limits.

Second, namespaces introduce some subtle differences in the details of applications inside containers. For example, tools like jcmd and jstack rely on some information provided by “/proc//” but Docker’s design changes the original structure of this information. We need to modify the original tools to adapt to this change.

From the perspective of JVM execution mechanism, why do these “communication barriers” lead to issues like OOM?

You can think about how this problem reflects how the JVM sets default parameters at startup based on the system’s resources (memory, CPU, etc.).

This is known as the Ergonomics mechanism. For example:

The JVM roughly sets the initial heap size at startup as 1/64 of the detected memory size and sets the maximum heap size as 1/4 of the system memory.
The CPU count detected by the JVM directly affects the number of parallel threads for Parallel GC and JIT compiler threads, and even the parallelism level of mechanisms like ForkJoinPool in our application.

These default parameters are chosen based on common scenarios. However, due to the differences in container environments, Java’s judgments are likely based on incorrect information. It’s like thinking I live in a whole villa, but in reality, I only have one room to live in.

More importantly, some existing diagnostic or fallback mechanisms of the JVM are also affected. To ensure service availability, a common approach is to rely on the “-XX:OnOutOfMemoryError” feature to call a processing script to take remedial measures, such as automatic service restart. However, this mechanism is implemented based on fork, and when the Java process has already committed excessive memory, it is often impossible for a new process to run normally.

Based on the previous summary, it seems that the problem is quite challenging. So, in practice, how do we solve these problems?

Firstly, if you can upgrade to the latest version of JDK, this problem can be solved.

For this situation, JDK 9 introduced some experimental parameters to facilitate “communication” between Docker and Java. For example, for memory limits, you can use the following parameters:

-XX:+UnlockExperimentalVMOptions
-XX:+UseCGroupMemoryLimitForHeap

Note that these two parameters are order-sensitive and only supported in Linux environments. As for limiting the number of CPU cores, Java has been fixed to correctly understand settings like “–cpuset-cpus” without the need for separate parameters.

If you can switch to JDK 10 or newer versions, the problem becomes even simpler. Java’s support for containers (Docker) is already quite mature, and it automatically adapts to various resource limits and implementation differences. The experimental parameter “UseCGroupMemoryLimitForHeap” mentioned earlier has been deprecated.

At the same time, new parameters were added to explicitly specify the number of CPU cores.

-XX:ActiveProcessorCount=N

If you encounter any issues in practice, you can also use “-XX:-UseContainerSupport” to disable the container support feature of Java. This can be a defensive mechanism to prevent new features from interfering with existing basic functionality. Of course, you are also welcome to provide feedback to the OpenJDK community.

Fortunately, the experimental improvements in JDK 9 have been backported to Oracle JDK 8u131. You can directly download the corresponding image and configure “UseCGroupMemoryLimitForHeap”. It is highly likely that further enhancements from JDK 10 will also be applied to the latest updates of JDK 8.

But what if I can only use an older version of JDK for now?

Here are a few suggestions:

Set the sizes of heap, metaspace, and other memory regions explicitly to ensure the total size of the Java process is controllable.

For example, we can limit the container memory in the environment as follows:

$ docker run -it --rm --name yourcontainer -p 8080:8080 -m 800M repo/your-java-container:openjdk

In this case, you can additionally configure the following environment variable to directly specify the JVM heap size:

-e JAVA_OPTIONS='-Xmx300m'

Configure the number of parallel GC and JIT threads to avoid excessive consumption of computational resources:

-XX:ParallelGCThreads
-XX:CICompilerCount

Besides the previously mentioned issues such as OOM, it has been observed in many scenarios that Java in a Docker environment may unexpectedly use Swap. The specific reason is yet to be determined, but it is likely caused by the failure of the Ergonomics mechanism. I suggest configuring the following parameters to clearly inform the JVM of the system memory limit:

-XX:MaxRAM=`cat /sys/fs/cgroup/memory/memory.limit_in_bytes`

You can also specify Docker runtime parameters, for example:

--memory-swappiness=0

This is influenced by the operating system’s Swappiness mechanism. When the memory consumption reaches a certain threshold, the operating system will try to swap out inactive processes. The above parameter explicitly disables swapping. Therefore, as we can see, the use of Java in Docker involves the comprehensive application of knowledge from the operating system, kernel, to JVM itself.

Looking back at my introduction to the JVM memory regions in Column 25, we can see that the memory consumption of the JVM is not limited to just the heap. Often, configuring only Xmx is not enough, and MaxRAM also helps JVM allocate other memory regions reasonably. If the application requires more Java startup parameters but you are unsure of appropriate values, you can try some community-provided tools. However, be aware of the limitations of general-purpose tools.

Furthermore, regarding the issue of container image size, if you are using JDK 9 or later versions, you can fully customize the minimal dependent Java runtime environment using the jlink tool, reducing the size of the JDK to tens of megabytes, making it easier to run.

Starting from the problems that may arise with Java in a Docker environment, I have analyzed why the container environment is not transparent to applications and how this deviation interferes with JVM-related mechanisms. Finally, starting from practical experience, I have introduced the solution approach to the main issues. I hope it will be helpful to you in practical development.

Practice Exercise #

Have you grasped the topic we discussed today? Today’s thinking question is: What methods can improve Java’s performance in scenarios such as microservices and serverless, which I mentioned earlier, where Java has shown its shortcomings?

Please write your thoughts on this question in the comment section. I will select the comments that have been deeply considered and send you a learning reward coupon. Feel free to join the discussion with me.

Are your friends also preparing for interviews? You can “invite a friend to read” by sharing today’s topic with them. Perhaps you can help them.