08 Why Multithreading Can Lead to Performance Problems #

In this lesson, we mainly learn why multithreading can cause performance issues.

What is a performance issue #

In the previous lesson, we learned about the thread safety issues caused by multithreading. However, for multithreading, it not only brings thread safety problems, but also may bring performance issues. You may wonder, isn’t the main purpose of using multithreading to improve performance? To let multiple threads work at the same time, speeding up program execution. Why would it instead bring performance issues?

This is because a single-threaded program works independently and does not need to interact with other threads. However, multiple threads need to be scheduled and cooperate with each other, and this scheduling and cooperation will incur performance overhead and generate performance issues.

Firstly, let’s understand what exactly is a performance issue? In fact, performance issues can take many forms, such as slow server response, low throughput, and excessive memory usage. We design excellent system architecture, purchase more CDN servers, and buy larger bandwidth to improve performance and enhance user experience. Although slow running speed may not have serious consequences, usually all we need to do is wait a few more seconds, it will significantly impact the user experience.

Research has shown that for every additional second it takes for a page to respond, at least 7% of users will be lost, and if it takes more than 8 seconds to return a result, almost all users will not choose to continue waiting. One of the important reasons for introducing multithreading is to improve program performance, so we cannot reverse the priority. We cannot make the program run slower just because multithreading is introduced. Therefore, we must solve the performance issues caused by multithreading.

Why does multithreading cause performance issues #

So, in what situations does multithreading programming cause performance issues? There are mainly two aspects: thread scheduling and thread coordination.

Overhead of Scheduling #

Context Switching #

First, let’s take a look at thread scheduling. In practical development, the number of threads is often greater than the number of CPU cores. For example, the number of CPU cores may be 8 or 16, but the number of threads can reach hundreds or even thousands. In this case, the operating system will allocate time slices to each thread based on certain scheduling algorithms, allowing each thread to have a chance to run. However, scheduling will trigger context switching, which involves suspending the currently executing thread, saving its current state, finding the next code to be resumed, and waking up the next thread, and so on. But context switching comes with a relatively high overhead. For example, if our task is very short, such as performing simple calculations, it is possible that the performance overhead caused by context switching is even greater than the performance overhead caused by executing the thread itself.

Cache Invalidation #

Not only does context switching cause performance issues, but cache invalidation can also lead to performance problems. Since there is a high probability that the program will access the data that was just accessed, a cache is used to speed up the entire program’s execution, making it possible to quickly retrieve the data when using the same data again. However, once a thread scheduling occurs and switches to another thread, the CPU will execute different code, thereby invalidating the original cache. The new data needs to be cached again, which also incurs overhead. Therefore, to avoid frequent context switching, the thread scheduler usually sets a minimum execution time for the scheduled thread, which means that only after this period of time has been executed, can the next scheduling possibly occur, thereby reducing the number of context switches.

So, what situations can cause intensive context switching? If the program frequently competes for locks or experiences frequent blocking due to IO operations, it may require more context switching, resulting in greater overhead. We should try to avoid such situations from happening.

Cost of Collaboration #

In addition to thread scheduling, thread collaboration can also potentially introduce performance problems. Because if there are shared data between threads, in order to avoid data corruption and ensure thread safety, it may be necessary to prohibit compiler and CPU optimizations such as reordering. It may also repeatedly flush the data in the thread’s working memory to the main memory for synchronization purposes, and refresh it from the main memory to the working memory of other threads, and so on. These problems do not exist in single-threaded programs, but in multi-threaded programs, these methods have to be adopted to ensure data correctness, because the priority of thread safety is higher than that of performance, which indirectly reduces our performance.