08 Actions Performed by Redis Server After Startup #
Starting from this lesson, we have entered the second module of the course. In this module, I will guide you to understand and learn about the knowledge related to the running of Redis instances, including the startup process of Redis server, the network communication mechanism based on event-driven framework, and the thread execution model of Redis. Today, let’s first learn about the startup process of the Redis server.
We know that the main function is the entry point of the entire Redis program, and when a Redis instance is running, it will start executing from this main function. At the same time, since Redis is a typical Client-Server architecture, once the Redis instance starts running, the Redis server will also start, and the main function is actually responsible for the startup and operation of the Redis server.
In Lesson 1, I introduced the overall architecture of the Redis source code to you. The basic control logic for Redis operation is implemented in the server.c file, and the main function is also in server.c.
When designing or implementing a network server program, you may encounter a question of what operations should be performed and whether there is a typical reference implementation when the server starts up. So in today’s lesson, I will start with the main function to introduce how the Redis server starts up and completes initialization. By learning the content of this lesson, you will be able to grasp the implementation ideas of Redis for the following three questions:
- What specific initialization operations does the Redis server perform after startup?
- What are the key configuration items during the Redis server initialization?
- How does the Redis server start processing client requests?
Moreover, the startup process of Redis server’s design and implementation also has certain representativeness. After learning it, you can generalize the key operations and apply them in your own network server implementation.
Alright, next, let’s start with the main function to understand its design and implementation in the Redis server.
main function: The entry point of Redis server #
Generally speaking, the code logic for starting and running a system software developed in C is implemented in the main function. Therefore, before delving into the implementation of the main function in Redis, I’d like to share a small tip with you. When reading and learning the code of a system, you can start by looking for the main function to understand its execution process.
For the main function of Redis, I divide its execution into five stages.
Stage 1: Basic Initialization
In this stage, the main function mainly performs some basic initialization work, including setting the time zone of the server and setting the random seed of the hash function. The main function calls the following functions for these tasks:
// Set time zone
setlocale(LC_COLLATE,"");
tzset();
...
// Set random seed
char hashseed[16];
getRandomHexChars(hashseed, sizeof(hashseed));
dictSetHashFunctionSeed((uint8_t*)hashseed);
Here, you need to pay attention to the overridden code block at the beginning of the main function. This block of code is used to check if the REDIS_TEST macro is defined and if the Redis server is started with the test parameters. If these conditions are met, the main function will execute the corresponding test program.
The code for this macro definition is shown below. The example code calls the test function for ziplist, ziplistTest:
#ifdef REDIS_TEST
// If the test and ziplist parameters are specified, call the ziplistTest function for ziplist testing
if (argc == 3 && !strcasecmp(argv[1], "test")) {
if (!strcasecmp(argv[2], "ziplist")) {
return ziplistTest(argc, argv);
}
...
}
#endif
Stage 2: Check for sentinel mode and perform RDB or AOF checks
After starting, the Redis server may run in sentinel mode, which differs from running in normal mode in terms of parameter initialization, parameter settings, and operations during server startup. Therefore, the main function needs to check if sentinel mode is set based on the Redis configuration.
If sentinel mode is set, the main function will call the initSentinelConfig function to initialize the parameters for sentinel mode and call the initSentinel function to initialize the server running in sentinel mode. I will give you a detailed introduction to the Redis server running in sentinel mode in Lesson 21.
The following code shows the check for sentinel mode in the main function, as well as the initialization for sentinel mode:
...
// Check if the server is set to sentinel mode
if (server.sentinel_mode) {
initSentinelConfig(); // Initialize sentinel configuration
initSentinel(); // Initialize sentinel mode
}
...
In addition to checking for sentinel mode, the main function also checks if RDB checking or AOF checking should be performed. This corresponds to running the redis-check-rdb or redis-check-aof program. In this case, the main function will call the redis_check_rdb_main or redis_check_aof_main function to check the RDB or AOF file. You can see the code below, which shows the check and call in the main function:
...
// If running redis-check-rdb program, call redis_check_rdb_main function to check RDB file
if (strstr(argv[0], "redis-check-rdb") != NULL)
redis_check_rdb_main(argc, argv, NULL);
// If running redis-check-aof program, call redis_check_aof_main function to check AOF file
else if (strstr(argv[0], "redis-check-aof") != NULL)
redis_check_aof_main(argc, argv);
...
Stage 3: Command line argument parsing
In this stage, the main function parses the command line arguments and calls the loadServerConfig function to merge and process the command line parameters and parameters in the configuration file. It then sets appropriate values for key parameters of the Redis modules to ensure efficient server operation.
Stage 4: Initialize the server
After parsing and setting the running parameters, the main function calls the initServer function to initialize various resources required for server operation. This mainly includes initializing data structures for server resource management, initializing the key-value database, and initializing the server network framework.
After calling initServer, the main function again checks if the current server is running in sentinel mode. If it is in sentinel mode, the main function calls the sentinelIsRunning function to set up the sentinel mode. Otherwise, the main function calls the loadDataFromDisk function to load the AOF or RDB file from disk to restore the previous data.
Stage 5: Execute the event-driven framework
In order to efficiently handle high-concurrency client connection requests, Redis uses an event-driven framework to concurrently handle connection and read/write requests from different clients. Therefore, when the main function reaches the end, it calls the aeMain function to enter the event-driven framework and start looping through various triggered events.
I have summarized the key operations involved in the five stages introduced earlier in the following diagram for your reference:
Among these five stages, Stages 3, 4, and 5 actually include the key operations during the startup process of Redis server. So next, we will learn about the main tasks in these three stages one by one.
Redis Configuration Parameters Analysis and Settings #
We know that Redis provides rich functionality, supporting read and write access to various key-value data types, as well as features such as data persistence, master-slave replication, and sharded clusters. The efficient operation of these features actually relies on the key parameter configurations of the corresponding modules.
For example, in order to save memory, Redis has designed memory compact data structures to store Hash, Sorted Set, and other key-value data types. However, after using memory compact data structures, if the number of elements stored in the data structure is too large or the size of the elements is too large, the performance of key-value access will be affected. Therefore, in order to balance memory usage and system performance, we can use parameters to set and adjust the conditions for using memory compact data structures.
In other words, mastering the settings of these key parameters can help us improve the efficiency of Redis instances.
However, there are many parameters in Redis, and we cannot master all the parameter settings in one lesson. Therefore, below, we can start by learning about the main parameter types in Redis, so that we can have a comprehensive understanding of various parameters. At the same time, I will also introduce you to some parameters closely related to server operation and their settings methods, so that you can configure these parameters to make the server operate efficiently.
Main Parameter Types in Redis #
First of all, all the parameters required for Redis operation are defined in the redisServer structure in the server.h file. Based on the scope of parameter effects, I divided various parameters into seven types, including general parameters, data structure parameters, network parameters, persistence parameters, master-slave replication parameters, sharded cluster parameters, and performance optimization parameters. You can refer to the content in the table below for details.
By doing this, if you can classify Redis parameters according to the above classification method, you will find that these parameters are actually corresponding to the main functional mechanisms of Redis. Therefore, if you want to have a deep understanding of typical configuration values of these parameters, you need to have an understanding of the working principles of the corresponding functional mechanisms. In the following lessons, I will also introduce you to the typical parameter configurations corresponding to the design of Redis functional modules.
Okay, now we have learned about the seven parameter types in Redis and their basic scope of functions. Next, let’s continue to learn how Redis sets these parameters.
Setting Methods for Redis Parameters #
Redis actually goes through three rounds of assignment for setting runtime parameters, namely default configuration values, command-line startup parameters, and configuration file values.
First of all, Redis will call the initServerConfig function in the main function to set default values for various parameters. The default values for parameters are uniformly defined in the server.h file and are macro-defined variables starting with CONFIG_DEFAULT. The following code shows the default values of some parameters, you can take a look.
#define CONFIG_DEFAULT_HZ 10 // Default running frequency for server background tasks
#define CONFIG_MIN_HZ 1 // Minimum running frequency for server background tasks
#define CONFIG_MAX_HZ 500 // Maximum running frequency for server background tasks
#define CONFIG_DEFAULT_SERVER_PORT 6379 // Default TCP port for server listening
#define CONFIG_DEFAULT_CLIENT_TIMEOUT 0 // Client timeout period, default is 0, indicating no timeout limit
The default parameter values provided in server.h are generally typical configuration values. Therefore, if you are not very familiar with the working principles of Redis various functional modules during the deployment and usage of Redis instances, you can use the default configurations provided in the code.
Of course, if you are familiar with the working mechanisms of Redis functional modules, you can also set the runtime parameters yourself. You can set the runtime parameter values on the command line when starting the Redis program. For example, if you want to change the listening port of the Redis server from the default 6379 to 7379, you can set the port parameter to 7379 on the command line as shown below:
./redis-server --port 7379
Here, you need to pay attention that Redis command-line parameter settings need to use two hyphens “–” to indicate the corresponding parameter name, otherwise Redis will not recognize the set runtime parameters.
After using the initServerConfig function to set default configuration values for parameters, the main function of Redis will then parse the command-line parameters passed to the Redis program one by one.
The main function will save the parsed parameters and their values as strings. Then, the main function will call the loadServerConfig function to perform the second and third rounds of assignment. The following code shows the parsing of command-line parameters in the main function and the process of calling the loadServerConfig function, you can take a look.
int main(int argc, char **argv) {
…
// Save command-line parameters
for (j = 0; j < argc; j++) server.exec_argv[j] = zstrdup(argv[j]);
…
if (argc >= 2) {
…
// Parse each runtime argument
while(j != argc) {
…
}
…
//
loadServerConfig(configfile,options);
}
Here, what you need to know is that the loadServerConfig function is implemented in the config.c file. This function takes the Redis configuration file and the parsing string of command-line arguments as parameters and reads all the configuration items from the configuration file, forming a string. Then, the loadServerConfig function appends the parsed command-line arguments to the configuration item string formed by the configuration file.
This way, the configuration item string contains both the parameters set in the configuration file and the parameters set via the command line.
Finally, the loadServerConfig function will further invoke the loadServerConfigFromString function to match each configuration item in the configuration item string. Once a match is found, the loadServerConfigFromString function will set the corresponding parameter of the server according to the value of the configuration item.
The following code shows part of the loadServerConfigFromString function. This part of the code uses conditional branches to compare whether the configuration item is “timeout” or “tcp-keepalive” one by one. If it matches, the server parameter is set to the value of the configuration item.
At the same time, the code checks whether the value of the configuration item is reasonable, such as whether it is less than 0. If the parameter value is not reasonable, the program will report an error at runtime. In addition to other configuration items, the loadServerConfigFromString function continues to use else if branches for judgment.
loadServerConfigFromString(char *config) {
…
// Match parameter name, check if the parameter is "timeout"
if (!strcasecmp(argv[0],"timeout") && argc == 2) {
// Set server's maxidletime parameter
server.maxidletime = atoi(argv[1]);
// Check if the parameter value is less than 0, report an error if it is less than 0
if (server.maxidletime < 0) {
err = "Invalid timeout value"; goto loaderr;
}
}
// Match parameter name, check if the parameter is "tcp-keepalive"
else if (!strcasecmp(argv[0],"tcp-keepalive") && argc == 2) {
// Set server's tcpkeepalive parameter
server.tcpkeepalive = atoi(argv[1]);
// Check if the parameter value is less than 0, report an error if it is less than 0
if (server.tcpkeepalive < 0) {
err = "Invalid tcp-keepalive value"; goto loaderr;
}
}
…
}
Okay, by now, you should have understood the steps for configuring runtime parameters of the Redis server. I have also created a diagram to help you better visualize this process.
After completing the configuration, the main function will start calling the initServer function to initialize the server. So, next, let’s continue to understand the key operations during the initialization of the Redis server.
initServer: Initialize Redis server #
The initialization of the Redis server can be divided into three steps.
- Step 1: When the Redis server is running, it needs to manage various resources.
For example, clients connected to the server, slave servers, the candidate set used for cache replacement, and runtime status information of the server. The initialization of these resource management information is done in the initServer function.
Let me give you an example. The initServer function creates linked lists to separately manage clients and slave servers. It also calls the evictionPoolAlloc function (in evict.c) to generate a candidate set of keys for eviction. Additionally, the initServer function calls the resetServerStats function (in server.c) to reset the runtime status information of the server.
- Step 2: After completing the initialization of resource management information, the
initServerfunction initializes the Redis databases.
Since a Redis instance can run multiple databases, the initServer function uses a loop to create the corresponding data structures for each database.
The following code logic is implemented in the initServer function. It initializes each database by creating global hash tables and tables for expired keys, keys blocked by BLPOP, keys to be pushed, and keys being watched.
for (j = 0; j < server.dbnum; j++) {
// Create global hash table
server.db[j].dict = dictCreate(&dbDictType, NULL);
// Create table for expired keys
server.db[j].expires = dictCreate(&keyptrDictType, NULL);
// Create table for keys blocked by BLPOP
server.db[j].blocking_keys = dictCreate(&keylistDictType, NULL);
// Create table for keys to be pushed
server.db[j].ready_keys = dictCreate(&objectKeyPointerValueDictType, NULL);
// Create table for keys being watched by MULTI/WATCH operations
server.db[j].watched_keys = dictCreate(&keylistDictType, NULL);
...
}
- Step 3: The
initServerfunction creates an event-driven framework for the running Redis server and starts listening on a port to receive external requests.
To efficiently handle high-concurrency external requests, the initServer function creates listening events for possible client connections on each listening IP. These events are used to listen for client connection requests. Additionally, the initServer function sets the corresponding handler function acceptTcpHandler for the listening events.
In this way, whenever a client connects to the IP and port that the server is listening on, the event-driven framework detects the connection event and calls the acceptTcpHandler function to handle the specific connection. You can refer to the following code for the handling logic:
// Create the event loop framework
server.el = aeCreateEventLoop(server.maxclients + CONFIG_FDSET_INCR);
...
// Start listening on the configured network port
if (server.port != 0 &&
listenToPort(server.port, server.ipfd, &server.ipfd_count) == C_ERR)
exit(1);
...
// Create timed events for server background tasks
if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) {
serverPanic("Can't create event loop timers.");
exit(1);
}
...
// Set the handler function acceptTcpHandler for each listening IP
for (j = 0; j < server.ipfd_count; j++) {
if (aeCreateFileEvent(server.el, server.ipfd[j], AE_READABLE,
acceptTcpHandler, NULL) == AE_ERR) { ... }
}
After the Redis server has completed the runtime parameter settings and initialization, it can start handling client requests. To be able to continuously handle concurrent client requests, the server enters an event-driven loop mechanism at the end of the main function. This is what we will learn about in the next section: the execution process of the event-driven framework.
Execution of Event-Driven Framework #
The event-driven framework is the core of the Redis server. Once the framework is launched, it continuously loops and processes a batch of triggered network read/write events in each iteration. I will provide specific information about the design principles and implementation methods of the event-driven framework in Lectures 9 to 11. In this lesson, we mainly focus on how the Redis entry point main function transitions to the event-driven framework for execution.
Entering and executing the event-driven framework is actually not complicated. The main function directly calls the core function aeMain (in the ae.c file) of the event framework, which then enters the event handling loop.
However, before entering the event-driven loop, the main function will call two functions, aeSetBeforeSleepProc and aeSetAfterSleepProc, respectively, to set the operations that the server needs to perform before entering each event loop, and the operations that the server needs to perform after each event loop. The following code snippet shows the execution logic of this part, which you can take a look at.
int main(int argc, char **argv) {
...
aeSetBeforeSleepProc(server.el, beforeSleep);
aeSetAfterSleepProc(server.el, afterSleep);
aeMain(server.el);
aeDeleteEventLoop(server.el);
...
}
Summary #
In today’s lesson, we have learned about the five main stages of the Redis server startup process through the design and implementation of the main function in the server.c file. Among these five stages, the parameter parsing, server initialization, and execution of the event-driven framework are the three key stages of the Redis server startup process. Therefore, we need to focus on the following three key points.
First, the main function uses initServerConfig to set default values for the server runtime parameters. It then parses the command line arguments and reads the configuration file parameters through loadServerConfig, appending the command line arguments to the configuration string. Finally, Redis calls loadServerConfigFromString to complete the setting of the configuration file parameters and command line arguments.
Second, after the Redis server completes the parameter setting, the initServer function is called to initialize the main data structures for server resource management. It also initializes the database startup status and sets the IP and port for server listening.
Third, once the server is able to receive requests from external clients, the main function hands over the control to the entry function of the event-driven framework, which is aeMain function. The aeMain function will be executed in a loop, processing incoming client requests. At this point, the functionality of the main function in server.c is completed, and the program control is handed over to the event-driven loop framework, allowing Redis to handle client requests normally.
In fact, the startup process of the Redis server, from basic initialization operations to command line and configuration file parameter parsing and setting, to the initialization of various server data structures, and finally the execution of the event-driven framework, is a typical process for network servers. When you develop network servers, you can use it as a reference.
Furthermore, mastering the initialization operations in the startup process can also help you answer some doubts during usage. For example, whether Redis reads the RDB file first or the AOF file first when starting. If you understand the startup process of the Redis server, you can see from the loadDataFromDisk function that Redis server reads the AOF file first. If there is no AOF file, then it reads the RDB file.
Therefore, mastering the Redis server startup process helps you better understand the running details of Redis, so that when you encounter problems, you know that you can trace back to the various initial states of the server from the startup process, which helps you better solve problems. 这段代码判断是否需要在后台运行 Redis。以下是代码的解释：
- 首先，
redisIsSupervised()函数用于判断 Redis 是否以受监管的方式运行。 - 然后，
server.daemonize变量表示是否需要以守护进程的方式运行 Redis。 server.supervised变量则表示 Redis 是否受监管。
综合上述三个条件，如果 background 变量为真，则调用 daemonize() 函数，将 Redis 设置为后台运行。
因此，这段代码的作用是判断 Redis 是否需要在后台运行，并在需要的情况下将其设置为后台运行。