03 Why High-Performance IO Model Makes Single-threaded Redis So Fast #
Today, we are going to discuss a question that many people are concerned about: “Why is single-threaded Redis so fast?”
First of all, it is important to clarify a fact. When we say that Redis is single-threaded, we are mainly referring to the fact that Redis handles network IO and key-value read/write operations with a single thread. This is the main process through which Redis provides key-value storage services. However, Redis does have additional threads for other functionalities such as persistence, asynchronous deletion, and cluster data synchronization.
Strictly speaking, Redis is not truly single-threaded. However, we generally refer to Redis as a single-threaded high-performance system because it sounds cooler. This will also lead you to the next question: “Why use a single thread? Why is a single-threaded system so fast?”
To understand this question, we need to delve into the design mechanism and multiplexing mechanism of Redis’s single-threaded model. This will also help you better optimize Redis performance by avoiding operations that may block the single thread, such as executing high-complexity commands.
Alright, without further ado, let’s dive into the reasons why Redis adopts a single-threaded model.
Why does Redis use a single thread? #
To better understand why Redis uses a single thread, we first need to understand the overhead of multithreading.
Overhead of multithreading #
When writing programs, we often hear the statement: “Using multiple threads can increase system throughput or improve system scalability.” Indeed, for a multithreaded system, with proper resource allocation, it can increase the number of resource entities that handle requests in the system, thereby improving the system’s ability to handle multiple requests simultaneously, i.e., throughput. The left figure below shows the expected result when using multiple threads.
However, please note that in most cases, when we use multiple threads without proper system design, the actual result is as shown in the right figure. When we initially increase the number of threads, the system throughput will increase. But when we further increase the number of threads, the system throughput will grow slower, and sometimes even decrease.
Number of threads and system throughput
Why does this happen? A key bottleneck is that there are usually shared resources that are accessed by multiple threads simultaneously in the system, such as a shared data structure. When multiple threads need to modify this shared resource to ensure its correctness, additional mechanisms are required to guarantee it, which in turn introduces additional overhead.
Take Redis as an example. In the previous lesson, I mentioned that Redis has the List data type and provides enqueue (LPUSH) and dequeue (LPOP) operations. Assuming Redis adopts a multithreaded design, as shown in the diagram below, there are two threads, A and B. Thread A performs an LPUSH operation on a List and increases the queue length by 1. At the same time, thread B performs an LPOP operation on the same List and decreases the queue length by 1. To ensure the correctness of the queue length, Redis needs to execute the LPUSH and LPOP operations of threads A and B in a serial manner, so that Redis can accurately record their modifications to the List’s length. Otherwise, we may obtain incorrect length results. This is the concurrent access control problem faced by the multithreaded programming model.
Concurrent access to Redis using multithreading
Concurrent access control has always been a challenging problem in multithreaded development. Without careful design, such as simply using a coarse-grained mutual exclusion lock, undesirable results will occur. Even with increased threads, most threads will be waiting to acquire the lock for accessing the shared resource, and parallelism turns into serialization, without an increase in system throughput.
Furthermore, when developing with multiple threads, synchronization primitives are usually introduced to protect the concurrent access to shared resources, which also reduces the code’s debuggability and maintainability. To avoid these issues, Redis directly adopts a single-threaded mode.
By now, you should understand “Why Redis uses a single thread”. Next, let’s take a look at why single-threaded Redis can achieve high performance.
Why is single-threaded Redis so fast? #
Generally speaking, single-threaded processing is much poorer than multi-threaded processing. However, Redis is able to achieve processing capability in the range of hundreds of thousands of operations per second using a single-threaded model. Why is that? In fact, this is a comprehensive result of Redis’ design choices from various aspects.
On one hand, most of Redis operations are completed in memory, and coupled with the use of efficient data structures such as hash tables and skip lists, this is an important reason for its high performance. On the other hand, Redis adopts the mechanism of multiplexing, which allows it to concurrently handle a large number of client requests in network IO operations, achieving high throughput. Next, let’s focus on learning about the multiplexing mechanism.
Firstly, we need to understand the basic IO model of network operations and the potential blocking points. After all, Redis performs IO with a single thread, so if the thread is blocked, multiplexing cannot be achieved.
Basic IO model and blocking points #
Do you remember the SimpleKV with a network framework that I introduced in the first lesson?
Taking Get request as an example, in order to process a Get request, SimpleKV needs to listen to client requests (bind/listen), establish a connection with the client (accept), read the request from the socket (recv), parse the client’s sent request (parse), read key-value data based on the request type (get), and finally return the result to the client by writing data back to the socket (send).
The figure below shows this process, where bind/listen, accept, recv, parse, and send belong to network IO processing, while get belongs to key-value data operations. Since Redis is single-threaded, the most basic implementation is to execute these operations in sequence within a single thread.
Redis basic IO model
However, there are potential blocking points in the network IO operations here, which are accept() and recv(). When Redis receives a connection request from a client but fails to establish the connection successfully, it will be blocked at the accept() function, preventing other clients from establishing a connection with Redis. Similarly, when Redis reads data from a client through recv() and the data does not arrive, Redis will also block at recv().
This results in the entire Redis thread being blocked and unable to process other client requests, resulting in low efficiency. However, fortunately, the socket network model itself supports non-blocking mode.
Non-blocking mode #
The non-blocking mode setting in the socket network model mainly reflects on three key function calls. If you want to use socket in non-blocking mode, you must understand the return types and mode settings of these three functions. Let’s focus on learning about them next.
In the socket model, different operations will return different socket types after calling. The socket() method will return an active socket, and then calling the listen() method will convert the active socket into a listening socket, which can then listen for connection requests from clients. Finally, calling the accept() method will accept incoming client connections and return the connected socket.
Redis socket types and non-blocking setting
Regarding the listening socket, we can set it to non-blocking mode: when Redis calls accept() but no connection request arrives, the Redis thread can return and process other operations instead of waiting indefinitely. However, it is important to note that the listening socket already exists when accept() is called.
Although the Redis thread doesn’t have to continue waiting, there should be a mechanism to continue waiting for subsequent connection requests on the listening socket and notify Redis when there is a request.
Similarly, we can also set the connected socket to non-blocking mode: after Redis calls recv(), if there is no data arriving on the connected socket, the Redis thread can return and process other operations. We also need a mechanism to continue monitoring the connected socket and notify Redis when data arrives.
This ensures that the Redis thread does not block at the blocking point like in basic IO models, and does not prevent Redis from processing actual connection requests or data that arrives.
At this point, the IO multiplexing mechanism in Linux comes into play.
High-performance I/O model based on IO multiplexing #
The IO multiplexing mechanism in Linux refers to a single thread processing multiple IO streams, which is commonly known as the select/epoll mechanism. Simply put, in the case of Redis running only on a single thread, this mechanism allows multiple listening sockets and connected sockets to coexist in the kernel. The kernel continuously listens for connection or data requests on these sockets. Once a request arrives, it is handed over to the Redis thread for processing, thus achieving the effect of a single Redis thread handling multiple IO streams.
The following figure shows the Redis IO model based on IO multiplexing. The multiple FDs in the figure are the multiple sockets mentioned earlier. The Redis network framework calls the epoll mechanism to let the kernel listen on these sockets. At this time, the Redis thread does not block on a specific listening or connected socket, which means it does not block on a specific client request processing. Because of this, Redis can connect and process requests from multiple clients at the same time, thereby improving concurrency.
High-performance Redis IO model based on IO multiplexing
In order to notify the Redis thread when a request arrives, select/epoll provides an event-based callback mechanism, which calls the corresponding processing function for different events.
So how does the callback mechanism work? In fact, once select/epoll detects that there is a request on the FD, it triggers the corresponding event.
These events are put into an event queue, and the Redis single thread continuously processes this event queue. In this way, Redis does not need to constantly poll whether requests have actually occurred, which avoids wasting CPU resources. At the same time, when Redis processes the events in the event queue, it calls the corresponding processing functions, thus realizing event-based callbacks. Because Redis is always processing the event queue, it can respond to client requests in a timely manner, improving Redis’s response performance.
To help you understand, let me explain it further using the examples of connection requests and read data requests.
These two requests correspond to the Accept event and Read event, and Redis registers the accept and get callback functions for these two events, respectively. When the Linux kernel detects a connection request or a read data request, it triggers the Accept event and Read event. At this time, the kernel will callback the corresponding accept and get functions in Redis to process them.
It’s like patients going to a hospital for medical treatment. Before the doctor actually diagnoses the patient, each patient (equivalent to a request) needs to go through triage, temperature measurement, registration, etc. If all of these tasks are done by the doctor, the doctor’s efficiency will be very low. Therefore, hospitals have set up triage desks, which continuously handle these tasks before diagnosis (similar to the Linux kernel listening to requests) and then hand over to the doctor for actual diagnosis. In this way, even with a single doctor (equivalent to a Redis single thread), efficiency can be improved.
However, it is important to note that even if your application is deployed on different operating systems, the multiplexing mechanism still applies. Because there are many different implementations of this mechanism, there are implementations based on select and epoll on the Linux system, as well as implementations based on kqueue on FreeBSD and evport on Solaris. Therefore, you can choose the appropriate multiplexing implementation based on the actual operating system that Redis runs on.
Summary #
Today, we focused on the three questions about Redis threads: “Is Redis really single-threaded?” “Why use a single thread?” “Why is a single thread so fast?”
Now, we know that Redis being single-threaded means that it uses one thread for network IO and data read/write operations. One core reason for using a single thread is to avoid concurrency control issues that come with multi-threaded development. Single-threaded Redis can achieve high performance, closely related to the IO model of multiplexing, as it avoids potential blocking points in network IO operations such as accept() and send()/recv().
Now that you understand these points, you are ahead of many others. If you have friends who are still unclear about these questions, feel free to share this information with them to help resolve their confusion.
Additionally, I want to give you a heads-up that you may have noticed: in May 2020, Redis 6.0 stable version was released, which introduces a multi-threaded model. So, what is the relationship between this multi-threaded model and the IO model discussed in this lesson? Will it introduce complex concurrency control problems? How much improvement will it bring to Redis 6.0? I will provide you with specific explanations about these questions in the upcoming lessons.
One Question per Lesson #
In this lesson, I have a little question for you. In the “Basic IO Model of Redis” diagram, do you think there are any potential performance bottlenecks? Please feel free to write down your thoughts and answers in the comment section. Let’s discuss and exchange ideas together.