Java并发类库提供了几种线程池，分别有以下特点：

1. FixedThreadPool（固定线程池）：该线程池会创建固定数量的线程，不同于动态线程池，线程数量是固定的。适用于执行长期的任务，效果较好。

2. CachedThreadPool（缓存线程池）：该线程池会根据需要创建新线程，线程数量不固定。如果线程池中有可用线程，会利用现有线程执行任务；如果线程池中没有可用线程，则创建新线程执行任务。适用于执行短期任务的场景。

3. SingleThreadPool（单线程池）：该线程池只会创建一个线程来执行任务，即使任务中断也会重新创建线程来执行。适用于需要顺序执行的任务。

4. ScheduledThreadPool（定时线程池）：该线程池可以根据指定的时间间隔执行任务。适用于需要定时执行任务的场景。

每种线程池都有不同的特点，根据不同的场景和需求可以选择合适的线程池来管理线程并提高系统资源利用效率。

21 What kinds of thread pools are provided by the Java concurrency libraries and their characteristics #

Developers typically use the common thread pool creation methods provided by Executors to create thread pools with different configurations. The main differences lie in the different types of ExecutorService or the different initial parameters.

Currently, Executors provides 5 different thread pool creation configurations:

• newCachedThreadPool(): It is a thread pool used to handle a large number of short-term tasks. It has several distinct features: it tries to cache threads and reuse them; when no cached threads are available, it creates new worker threads; if a thread remains idle for more than 60 seconds, it is terminated and removed from the cache; when idle for a long time, this thread pool does not consume much resources. It uses a SynchronousQueue as the work queue internally.

• newFixedThreadPool(int nThreads): It reuses a specified number of threads (nThreads). It uses an unbounded work queue behind the scenes, so there can be up to nThreads active worker threads at any time. This means that if the number of tasks exceeds the number of active threads, they will wait in the work queue for idle threads to appear. If a worker thread exits, a new worker thread will be created to make up the specified number nThreads.

• newSingleThreadExecutor(): Its feature is that the number of worker threads is limited to 1, and it operates on an unbounded work queue. Therefore, it guarantees that all tasks are executed in order and there is at most one active task. The thread pool instance cannot be modified by the user, so it can prevent the number of threads from being changed.

• newSingleThreadScheduledExecutor() and newScheduledThreadPool(int corePoolSize): They create a ScheduledExecutorService for scheduling tasks at fixed rates or delays. The difference is whether there is a single worker thread or multiple worker threads.

• newWorkStealingPool(int parallelism): This is a thread pool that is often overlooked. It was added in Java 8. It internally builds a ForkJoinPool and uses the Work-Stealing algorithm to process tasks in parallel, without guaranteeing the processing order.

Focus Analysis #

The Executor framework in Java concurrency package is undoubtedly a focus in concurrent programming. Today’s question is about understanding several standard thread pools, and I provide an answer based on the most common usage.

In most application scenarios, using the 5 static factory methods provided by Executors is sufficient, but it may still be necessary to create a thread pool directly using constructors like ThreadPoolExecutor. This requires a further understanding of thread construction methods, and you need to understand the design and structure of thread pools.

In addition, the definition of thread pool is a term that can easily be misunderstood because ExecutorService provides more comprehensive thread management, task submission, and other methods besides the usual meaning of “pool”.

• The Executor framework is not just a thread pool. I think at least the following points are worth studying in depth:
• Understand the main content of the Executor framework, at least understand its components and responsibilities, and master its usage in basic development scenarios.
• Understand thread pools, related concurrency tool types, and even master their source code at a deeper level.
• Understand the common problems in practice, and have a basic diagnostic approach.
• Know how to use thread pools effectively according to the characteristics of your own application.

Knowledge Expansion #

First, let’s take a look at the basic components of the Executor framework, please refer to the class diagram below.

Let’s understand the main design purposes of each type from an overall perspective:

• Executor is a basic interface, and its purpose is to decouple task submission and task execution details. This can be understood from its definition of the only method.

void execute(Runnable command);

The design of Executor originates from the lessons learned from the early thread API used in Java. When developers implement application logic, they are disturbed by too many unrelated details such as thread creation and scheduling. Just like when we communicate over HTTP, if we still need to manipulate TCP handshake, the development efficiency is low and the quality is difficult to guarantee.

• ExecutorService is more comprehensive. It not only provides management functions of the service, such as shutdown method, but also provides a more comprehensive mechanism for submitting tasks, such as the submit method that returns a Future instead of void.

<T> Future<T> submit(Callable<T> task);

Note that in this example, we use a Callable as the input. It solves the problem that Runnable cannot return a result.

• The Java standard library provides several basic implementations, such as ThreadPoolExecutor, ScheduledThreadPoolExecutor, and ForkJoinPool. The design characteristics of these thread pools are their high adjustability and flexibility, in order to meet the complex and changing real-world application scenarios as much as possible. I will further analyze the source code of their construction parts and analyze the source of this flexibility.

• Executors, from the perspective of simplifying usage, provide us with various convenient static factory methods.

Now I will analyze the design and implementation of thread pools from the source code perspective. I will mainly focus on the source code of the most basic ThreadPoolExecutor. ScheduledThreadPoolExecutor is an extension of ThreadPoolExecutor, mainly adding scheduling logic. If you want to learn more about it, you can refer to the related tutorial. ForkJoinPool is a thread pool customized for ForkJoinTask, and it is different from the usual meaning of thread pool.

This part of the content is relatively obscure, listing concepts is not conducive for you to understand, so I will use some diagrams to illustrate. In the real application, understanding the interaction between the application and the thread pool and the internal workings of the thread pool, you can refer to the following diagram.

Let’s understand it briefly:

• The work queue is responsible for storing the various tasks submitted by users. This work queue can be a SynchronousQueue with a capacity of 0 (used by newCachedThreadPool), or it can be a LinkedBlockingQueue used by a fixed-size thread pool (newFixedThreadPool).

private final BlockingQueue<Runnable> workQueue;

• The internal “thread pool” refers to the collection of working threads that are maintained by the thread pool. The thread pool needs to manage thread creation and destruction during operation. For example, for a cached thread pool, when there is a heavy workload, the thread pool will create new working threads; when the business pressure recedes, the thread pool will end the idle threads after a period of time (default is 60 seconds).

private final HashSet<Worker> workers = new HashSet<>();

The working threads of the thread pool are abstracted as static inner class Workers and implemented based on AQS.

• ThreadFactory provides the logic for creating threads needed above.
• If a task is rejected when it is submitted, such as when the thread pool is already in SHUTDOWN state, a processing logic needs to be provided for it. The Java standard library provides default implementations such as ThreadPoolExecutor.AbortPolicy, and it can also be customized according to actual requirements.

From the above analysis, we can see several basic components of the thread pool, which are all reflected in the constructor of the thread pool. From the literal meaning, we can roughly guess its intention:

• corePoolSize, the so-called core thread count, can be roughly understood as the number of long-term resident threads (unless allowCoreThreadTimeOut is set). For different thread pools, this value may vary greatly. For example, newFixedThreadPool sets it to nThreads, while newCachedThreadPool sets it to 0.
• maximumPoolSize, as the name suggests, is the maximum number of threads that can be created when the threads are not enough. Make a comparison again, for newFixedThreadPool, of course, it is nThreads because it requires a fixed size, while for newCachedThreadPool, it is Integer.MAX_VALUE.
• keepAliveTime and TimeUnit, these two parameters specify how long the additional threads can be idle. Obviously, some thread pools do not need them.
• workQueue, the work queue, must be a blocking queue.

By configuring different parameters, we can create thread pools with very different behaviors. This is the basis for the high flexibility of thread pools.

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue,
                          ThreadFactory threadFactory,
                          RejectedExecutionHandler handler)

Further analysis reveals that the thread pool has a lifecycle and its state is represented in a specific way.

There is an interesting design here where the ctl variable serves a dual role. With the difference in the high and low bits, it represents both the state of the thread pool and the number of working threads. This is a typical efficient optimization. In actual systems, although we can specify the thread limit as Integer.MAX_VALUE, it is only a theoretical value due to resource limitations. Therefore, the idle bits can be assigned a different meaning.

private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
// The theoretical upper limit that determines the number of working threads
private static final int COUNT_BITS = Integer.SIZE - 3;
private static final int COUNT_MASK = (1 << COUNT_BITS) - 1;
// Thread pool state stored in the high bits of the number
private static final int RUNNING = -1 << COUNT_BITS;
...
// Packing and unpacking ctl
private static int runStateOf(int c)  { return c & ~COUNT_MASK; }
private static int workerCountOf(int c)  { return c & COUNT_MASK; }
private static int ctlOf(int rs, int wc) { return rs | wc; }

To give you a clearer impression of the thread lifecycle, I have drawn a simple state transition diagram here, corresponding to the possible states of the thread pool and its internal methods. If there are any methods that you don’t understand, please refer to the Javadoc. Note that the so-called Idle state does not actually exist in the Java code. I have added it only for better understanding.

So far, we have analyzed the thread pool attributes and construction aspects. Now, let’s take a look at a typical execute method to see how it works. Please refer to the comments I added for the specific logic, which will make it easier to understand when combined with the code.

public void execute(Runnable command) {
...
  int c = ctl.get();
  // Check the number of working threads, if lower than corePoolSize, add a Worker
  if (workerCountOf(c) < corePoolSize) {
      if (addWorker(command, true))
          return;
      c = ctl.get();
  }
  // isRunning checks whether the thread pool is shutdown
  // The work queue may be bounded, and offer is a friendly way of enqueueing
  if (isRunning(c) && workQueue.offer(command)) {
      int recheck = ctl.get();
  // Perform a defensive check again
      if (! isRunning(recheck) && remove(command))
          reject(command);
      else if (workerCountOf(recheck) == 0)
          ...

          addWorker(null, false);
}
// Try to add a worker. If failed, it means that the pool is already saturated or has been shut down.
else if (!addWorker(command, false)) {
    reject(command);
}

}

Practical use of thread pools

Although thread pools provide powerful and convenient functionality, they are not a silver bullet and can cause problems if used improperly. Here are some typical cases that can be deduced naturally based on the analysis:

• Avoid accumulating tasks. As I mentioned earlier, newFixedThreadPool creates a fixed number of threads, but its work queue is unbounded. If the number of worker threads is too small and cannot keep up with the rate at which tasks are being enqueued, it can consume a large amount of system memory and even lead to OOM. To diagnose this, you can use tools like jmap to check if there are a large number of task objects being enqueued.
• Avoid excessive thread expansion. When dealing with a large number of short-lived tasks, we usually use a cached thread pool, such as the default implementation in the latest HTTP/2 client API. When creating a thread pool, it is difficult to accurately estimate the amount of task pressure, or the characteristics of the data (are most requests 1K, 100K, or over 1M?). Therefore, it is difficult to set an exact number of threads.
• Additionally, if the number of threads keeps increasing (you can use tools like jstack to check), another possibility to be aware of is thread leakage, which often occurs when there is a problem with the task logic, causing worker threads to remain unreleased. I suggest you investigate the thread stack and it is likely that multiple threads are stuck at similar code locations.
• Avoid synchronization issues such as deadlocks. For scenarios and troubleshooting related to deadlocks, you can review [Column 18].
• Do try to avoid manipulating ThreadLocal when using thread pools. This has already been analyzed in [Column 17]. Through today’s understanding of thread pools, you should now better understand the reasons behind it, as the lifetimes of worker threads usually exceed the lifetimes of the tasks.

Choosing the thread pool size strategy

As mentioned earlier, an inappropriate thread pool size, whether too large or too small, can lead to trouble. Therefore, we need to consider a suitable thread pool size. Although it cannot be determined with certainty, there are some relatively universal rules and considerations.

• If our tasks are mainly computational in nature, it means that CPU processing power is a scarce resource. Can we increase computational capacity by increasing the number of threads? Often we can’t, as having too many threads may lead to a large overhead of context switching. In such cases, it is generally advisable to use the number of CPU cores N or N+1.

• If the tasks involve a significant amount of waiting, such as with a high I/O workload, you can refer to the calculation method recommended by Brian Goetz:

  Number of threads = CPU cores * target CPU utilization * (1 + average wait time / average work time)

These times cannot be accurately known in advance and need to be calculated based on sampling or summary analysis, which then needs to be validated and adjusted in practice.

• The above considerations only take into account CPU limitations. In reality, other system resource limitations may also have an impact. For example, I recently encountered a situation of limited ephemeral ports under heavy load on Mac OS X. Of course, I solved it by expanding the available port range. If we cannot adjust the capacity of the resource, then we can only limit the number of worker threads. The resources here can be file handles, memory, etc.

Also, in practical work, don’t rely solely on adjusting the thread pool as the solution to the problem. In many cases, architectural changes are more effective in solving the problem, such as using the backpressure mechanism of Reactive Stream and proper splitting.

Today, I explained and analyzed the main components of the Executor framework, such as the structure and lifecycle of thread pools, starting from the creation of thread pools in Java. I hope this has been helpful to you.

Practice #

Have you grasped the topic we discussed today? Today’s thought question is about understanding logically, thread pool creation, and lifecycle. Please discuss what values corePoolSize, maxPoolSize, etc. would be if you create a thread pool using newSingleThreadExecutor(). How many times might ThreadFactory be used in the lifecycle of the thread pool? How can you verify your judgment?

Please write your thoughts in the comments section. I will select well-thought-out comments and reward you with a learning coupon. Feel free to discuss with me.

Are your friends also preparing for interviews? You can “invite friends to read” and share today’s topic with them. Maybe you can help them.