15 How Many CPUs Core Number and Thread Count Are Appropriate #

In this lesson, we mainly study the appropriate number of threads and the relationship between CPU cores and the number of threads.

You may often be asked these two questions in interviews. In order to answer them well, you need to understand that the most important purpose of adjusting the number of threads in a thread pool is to fully and reasonably use CPU and memory resources, thereby maximizing the performance of the program. In practical work, we need to choose the corresponding strategy according to different types of tasks. #

CPU-bound tasks #

First of all, let’s look at CPU-bound tasks, such as encryption, decryption, compression, and calculation, which are a series of tasks that consume a lot of CPU resources. For these types of tasks, the best number of threads is 1 to 2 times the number of CPU cores. If you set too many threads, it will not be effective. Assuming we set the number of threads to be more than double the number of CPU cores, because the computational tasks are very heavy and occupy a large amount of CPU resources, each core of the CPU is basically fully loaded. When we set too many threads, each thread wants to use CPU resources to perform its own tasks, which will cause unnecessary context switching. Increasing the number of threads in this case does not improve performance. On the contrary, too many threads will lead to performance degradation.

For this kind of situation, it is best to consider at the same time which other programs on the same machine will consume too much CPU resources, and then balance the use of resources as a whole.

IO-bound tasks #

The second type of tasks is IO-bound tasks, such as database operations, file reading and writing, and network communication. These tasks do not consume much CPU resources, but the IO operations are time-consuming and will take up a lot of time in total. For this type of task, the maximum number of threads is usually much greater than the number of CPU cores. This is because IO read and write operations are slower compared to CPU speed. If we set too few threads, it may waste CPU resources. If we set more threads, when some threads are waiting for IO, they do not need CPU to do calculations, so other threads can use CPU to perform other tasks without affecting each other. This reduces the number of tasks in the task queue, which can better utilize resources.

The recommended calculation method by Brian Goetz, the author of “Java Concurrency in Practice”:

Number of threads = Number of CPU cores * (1 + average wait time/average work time)

Through this formula, we can calculate a reasonable number of threads. If the average wait time of the task is long, the number of threads will increase, and if the average work time is long, as in the case of our CPU-bound tasks mentioned above, the number of threads will decrease.

Too few threads will lower the overall performance of the program, and too many threads will consume other resources such as memory. Therefore, for more accurate results, load testing can be performed to monitor the thread situation of the JVM and the CPU load, and determine the number of threads to be created based on the actual situation to make more reasonable and efficient use of resources.

Conclusion #

In summary, we can draw a conclusion:

• The higher the proportion of the average work time of the threads, the fewer threads are needed;
• The higher the proportion of the average wait time of the threads, the more threads are needed;
• Perform corresponding actual tests for different programs to determine the most appropriate choice.