32 How to Implement Unit Testing in a System #

In today’s lesson, I’m going to talk to you about unit testing in Redis.

Unit testing is usually used to test a specific functional module of a system. Through unit testing, we can verify if the developed functional module is working correctly. For a system like Redis that contains many functional modules, unit testing becomes even more important. Otherwise, if we perform integrated testing directly after the whole system is developed, it will be difficult to locate problems once they occur.

So, how do we conduct unit testing in a system that contains multiple functional modules? In the Redis source code, a unit testing framework is provided for its main functional modules, such as operations on different data types, AOF and RDB persistence, master-slave replication, and clustering.

In today’s lesson, I will teach you about Redis’ implementation of the unit testing framework. By learning the content of today’s lesson, you will grasp how to develop a unit testing framework using the Tcl language. These testing development methods can also be applied to your daily development and testing work.

Next, let’s take a look at Redis’ unit testing framework implemented for its main functional modules.

Tcl Language Basics #

From Lesson 1 of the course, we learned that there is a dedicated “tests” directory in the Redis source code directory, which contains the implementation code of the Redis unit testing framework. Before understanding this unit testing framework, you first need to know that the framework is developed using the Tcl language.

Tcl stands for Tool Command Language. It is a feature-rich and easy-to-learn dynamic programming language that is often applied in scenarios such as program testing and operations management. Here, let me first introduce you to some basic knowledge and operations of Tcl language, and you can also learn more comprehensive development knowledge on the official website of Tcl language.

• Execution of Tcl Programs

Tcl language itself is an interpreted programming language, so the programs developed using Tcl do not need to be compiled and linked. Each statement is interpreted and executed.

• Data Types and Basic Operations

The data type in Tcl language is very simple, which is a string. We can use the set keyword to define variables without specifying the type of the variable. Also, we can use the puts keyword for output operations.

In terms of using variables, we need to understand two points: first, when outputting the value of a variable, we need to use the $ symbol to refer to the variable; second, we can use two colons to define a global variable, for example, ::testnum defines a global variable.

The following code shows the definition and output of variable a, where the value of variable a is defined as “hello tcl”.

set a "hello tcl"
puts $a

If you have the tclsh command interpreter installed on your computer, you can directly run tclsh on the command line, which will enter the Tcl command interpreter execution environment. If you don’t have it installed, you can download and install the source package from the Tcl official website.

Then, you can run the two statements mentioned above in the tclsh execution environment, as shown below:

tclsh    // Run the `tclsh` command, requires the `tclsh` command interpreter to be installed
// Enter the `tclsh` execution environment
% set a "hello tcl"
hello tcl
% puts $a 
hello tcl

Okay, what we just introduced are the basic operations for setting and outputting variables in Tcl. In addition to these, we can also define subroutines (procs) to perform frequently used functions. The following code demonstrates the definition of a proc subroutine:

proc sum {arg1 arg2} {
    set x [expr $arg1 + $arg2]
    return $x
}

From the code, you can see that the proc keyword is followed by the function name sum. Then, the function parameters arg1 and arg2 are enclosed in curly braces. The function body sets the value of variable x, which is equal to the sum of the two arguments arg1 and arg2.

Here, you need to note that in Tcl language, square brackets can enclose a command to execute that command and obtain a return result. Therefore, in the code mentioned earlier, [expr $arg1 + $arg2] represents the calculation of the sum of arg1 and arg2. Finally, this function returns the value of variable x, which also uses the $ symbol to refer to the variable x.

Now, we have learned some basic knowledge and operations of Tcl language. Next, let’s take a look at the Tcl-based unit testing framework used in Redis. Of course, during the process of learning the unit testing framework, I will also introduce some basic knowledge related to Tcl development so that you can understand the implementation of the testing framework.

Implementation of Redis Unit Testing Framework #

When using Redis unit testing framework, we need to execute the test script test_helper.tcl in the tests directory of the Redis source code, as shown below:

tclsh tests/test_helper.tcl

From here, you can see that the entry point of the unit testing framework is implemented in the test_helper.tcl file. Because Tcl is an interpreted language, when test_helper.tcl is executed, the statements in it will be sequentially interpreted and executed. However, you need to note that these statements are not proc subroutines, proc subroutines are meant to be called and executed. Now, let’s first understand the basic operations carried out when test_helper.tcl is executed.

Basic Operations after Running test_helper.tcl #

We can search for statements that do not start with “proc” in test_helper.tcl to understand the basic operations performed after running this script.

In reality, after running test_helper.tcl, the following three steps are mainly performed:

Step 1: Import other Tcl script files and define global variables

The test_helper.tcl script first uses the “source” keyword to import script files such as redis.tcl and server.tcl in the support subdirectory under the tests directory.

These script files implement some functionalities required by the unit testing framework. For example, in the server.tcl script file, the subroutine “start_server” is implemented to start a Redis testing instance, and the redis.tcl script implements the subroutine to send commands to the testing Redis instance.

Besides importing script files, the first step also includes defining global variables. For example, the testing framework defines a global variable ::all_tests, which contains all the predefined unit tests. If we run test_helper.tcl without any parameters, the testing framework will run all the tests defined by ::all_tests. Moreover, the global variables defined in the first step include the host IP, port number, skipped test case set, single test case set, and so on.

The following code shows part of the content executed in this step. You can take a look at it. You can also check all the imported scripts and defined global variables in the test_helper.tcl file.

source tests/support/redis.tcl
source tests/support/server.tcl
…

set ::all_tests {
unit/printver
unit/dump
unit/auth
…
}

set ::host 127.0.0.1
set ::port 21111
…
set ::single_tests {}  ; single test case set

After understanding the imported scripts and global variables, let’s take a look at the second step performed by the test_helper.tcl script, which is parsing script arguments.

Step 2: Parse script arguments

This step is a for loop that executes after the test_helper.tcl script imports other scripts and defines global variables.

The loop itself is not complicated. Its purpose is to sequentially parse the arguments carried when executing the test_helper.tcl script. However, to understand this flow, you need to have more knowledge about Tcl language development. For example, you need to know that the “llength” keyword is used to get the length of a list, and “lindex” is used to get an element from a list.

The following code shows the basic structure of this loop. Take a look at the comments, as they can help you gain more knowledge about Tcl language development.

for {set j 0} {$j < [llength $argv]} {incr j} {  ; use llength to get the length of parameter list argv
    set opt [lindex $argv $j]  ; get the j-th parameter from the argv parameter list using lindex
    set arg [lindex $argv [expr $j+1]]  ; get the j+1-th parameter from the argv parameter list
    if {$opt eq {--tags}} { …}     ; handle "--tags" parameter
    elseif {$opt eq {--config}} { …}  ; handle "--config" parameter
    …
}

Now, during the parsing of arguments, if the test_helper.tcl script has the “–single” parameter, it means that the script does not execute all test cases, but only one or multiple test cases. Therefore, the global variable ::single_tests in the script will save these test cases, and the ::all_tests global variable will be set to the value of ::single_tests, indicating that only the test cases in ::single_tests will be executed, as shown below:

if {[llength $::single_tests] > 0} {
…
}

set ::all_tests $::single_tests

Alright, after parsing the command line arguments, the third step of the test_helper.tcl script is to start the actual testing process.

• Step 3: Start the testing process

In this step, the test_helper.tcl script will check the value of the global variable ::client, which indicates whether to start the test client. If the value of ::client is 0, it means that the test client is not being started at the moment, so the test_helper.tcl script will execute the test_server_main function. Otherwise, the test_helper.tcl script will execute the test_client_main function. The logic is as follows:

if {$::client} {  // Start the test client
  if {[catch { test_client_main $::test_server_port } err]} { // Execute test_client_main
  …
  }
  else {  // Not starting the test client
     …
     if {[catch { test_server_main } err]} { …}  // Execute test_server_main
  }
}

I have drawn a diagram here to show the basic flow of execution of the test_helper.tcl script, you can review it again.

In fact, both the test_server_main and test_client_main functions are used to start the testing process. So what is their purpose? Let’s find out below.

test_server_main function #

The main tasks of the test_server_main function include three steps.

First, it starts a test server using the socket -server command. This test server creates a socket and listens for messages from the test client. Once a client connects, the test server will execute the accept_test_clients function. The code for this process is as follows:

socket -server accept_test_clients -myaddr 127.0.0.1 $port

For the accept_test_clients function, it calls the fileevent command to listen for read events when a client connects. If a read event occurs, it means that the client has sent a message to the test server. In this case, it will execute the read_from_test_client function. The process is as follows:

proc accept_test_clients {fd addr port} {
    …
    fileevent $fd readable [list read_from_test_client $fd]
}

The read_from_test_client function will execute different code branches based on the different messages sent by the test client. For example, when the test client sends the message “ready”, it means that the client is idle, so the test server can send unfinished test cases to this client for execution. This process is handled by the signal_idel_client function, you can read its source code for further understanding.

Similarly, when the message sent by the test client is “done”, the read_from_test_client function will count the number of completed test cases and also call the signal_idel_client function to let the current client continue executing unfinished test cases. You can read the code of the read_from_test_client function to understand the different code branches.

Alright, in the first step of the test_server_main function, it mainly starts the test server. The next second step is to start the test client.

The test_server_main function executes a for loop, in which it calls the exec command and executes the Tcl script one by one based on the number of test clients to be started. The number of test clients is determined by the global variable ::numclients, with a default value of 16. The executed Tcl script is the current test_helper.tcl script, with the same arguments as the current script, and an additional “–client” parameter to indicate that the test client is being started.

The following code shows the for loop described above.

for {set j 0} {$j < $::numclients} {incr j} {
   set start_port [find_available_port $start_port] // Set the port for the test client
   // Use the exec command to execute the test_helper.tcl script (script), with the same arguments as the current script, 
   // adding client parameter to indicate that the test client is being started; add port parameter to indicate the client port
   set p [exec $tclsh [info script] {*}$::argv \
            --client $port --port $start_port &]
   lappend ::clients_pids $p  // Record the process ID of each test client script execution
   incr start_port 10 // Increment the port number for the test client
}

Here, you need to pay attention. When the test_helper.tcl script is run with the “–client” parameter, it sets the global variable ::client to 1 during the parsing of the command-line arguments, as shown below:

for {set j 0} {$j < [llength $argv]} {incr j} {
   ...
   elseif {$opt eq {--client}} {
        set ::client 1
        ...
  }

As a result, when we execute this test_helper.tcl script in the previously mentioned loop flow, based on the value of the global variable ::client, it will actually start the testing client and execute the test_client_main function, as shown below:

if {$::client} {  # execute test_client_main function if ::client is 1
  if {[catch { test_client_main $::test_server_port } err]} {…}
}

So, after starting the testing client, the last step of the test_server_main function is to periodically execute the test_server_cron function every 10 seconds. The main task of this function is to output error messages and clean up the testing client and server when the test execution times out.

By now, you have understood the execution function test_server_main of the testing server, which mainly involves starting a socket to wait for client connections and handle client messages, as well as starting the testing client. The following diagram shows the basic flow of the test_server_main function for your reference.

Next, let’s take a look at the execution function test_client_main corresponding to the testing client.

test_client_main Function #

When the test_client_main function is executed, it first sends a “ready” message to the testing server. As I mentioned earlier, once the testing server detects a client connection that sends a “ready” message, it uses the signal_idle_client function to send the unfinished unit tests to this client.

Specifically, the signal_idle_client function sends a message like “run test_case_name” to the client. For example, if the current unfinished test case is unit/type/string, the signal_idle_client function will send the message “run unit/type/string” to the testing client. You can also take a look at the code below:

# Get the next untested case from ::all_tests and send a message to the client in the format of "run test_case_name"
send_data_packet $fd run [lindex $::all_tests $::next_test] 

After sending the “ready” message, the test_client_main function enters a while loop to wait for messages from the testing server. When it receives the “run test_case_name” message from the testing server, it calls the execute_tests function to execute the corresponding test case.

The following code shows the basic execution process of the test_client_main function mentioned earlier:

proc test_client_main fd {
…
    send_data_packet $::test_server_fd ready [pid]  # send ready message to the testing server
    while 1 {  # read the unit test messages sent from the testing server
        …
        set payload [read $::test_server_fd $bytes]  # read the message from the testing server
        foreach {cmd data} $payload break  # cmd represents the command sent by the testing server, and data represents the message content after the cmd command
        if {$cmd eq {run}} {  # if the message contains the "run" command
            execute_tests $data  # call execute_tests to execute the test case corresponding to data
        }
…
}

Now, let’s take a look at the execute_tests function, which is responsible for executing the test case. This function is quite simple. It uses the source command to import and execute the Tcl script file corresponding to the test case in the tests directory based on the passed-in test case name. Finally, it sends a “done” message to the testing server.

The relevant code is shown below:

proc execute_tests name {
    set path "tests/$name.tcl"  # find the corresponding test case file in the tests directory
    set ::curfile $path
    source $path  # import and execute the script file of the test case
}

```bash
send_data_packet $::test_server_fd done "$name" // After the test case is executed, send a "done" message to the test server
}

From here, we can see that during testing, the unit testing framework actually executes the TCL script file of each test case. This means that the test content for each test case is already written in its test script, and the framework executes the test script directly.

Next, let’s take a look at the implementation of the test case.

Implementation of Test Case #

In the Redis unit testing framework, there are many test cases, all of which are defined in the global variable ::all_tests. Here, we take the test case for the String data type, unit/type/string, as an example to understand the development implementation of the test cases in the framework.

The test script corresponding to the unit/type/string test case is string.tcl. This script first calls the start_server function to start a test Redis instance. The start_server function is defined in the server.tcl file, and you can further read the source code of this function to understand its implementation.

Then, the test script will test different test items separately. It uses the r function to send specific commands to the test Redis instance. For example, in the following code, the test script tests the SET and GET commands.

start_server {tags {"string"}} {
    test {SET and GET an item} {
        r set x foobar
        r get x
  } {foobar}
  …
}

Here, the function r (defined in the test_helper.tcl file) sends the test commands by calling the srv function (also defined in the test_helper.tcl file) to obtain the function named ::redis::redisHandle from the framework configuration.

The ::redis::redisHandle function is associated with the ::redis::dispatch function in the redis.tcl file, indicating that the ::redis::dispatch function will execute it. However, the ::redis::dispatch function will further call the ::redis::dispatch raw__ function to actually send the test commands.

Here, you need to pay attention that all three function names mentioned earlier will have an ID number. This ID number is dynamically assigned during the script execution and represents the socket descriptor of the test Redis instance to which the test commands are to be sent.

The following code shows the definition of the ::redis::redisHandle function association and the basic definition of the ::redis::dispatch function.

proc redis {{server 127.0.0.1} {port 6379} {defer 0}} {
…
interp alias {} ::redis::redisHandle$id {} ::redis::__dispatch__ $id
}
 
proc ::redis::__dispatch__ {id method args} {
  set errorcode [catch {::redis::__dispatch__raw__ $id $method $args} retval]
  …
}

So far, we know that the final actual sending of the test commands is done by the function ::redis::dispatch raw__. This function encapsulates the Redis commands in accordance with the RESP protocol and sends them to the test Redis instance. You can see the code below.

proc ::redis::__dispatch__raw__ {id method argv} {
set fd $::redis::fd($id)  //Get the socket descriptor of the test Redis instance to be sent
…
//Encapsulate Redis commands according to RESP protocol
set cmd "*[expr {[llength $argv]+1}]\r\n"  //Encapsulate the command and the number of parameters
append cmd "$[string length $method]\r\n$method\r\n" //Encapsulate the command name
foreach a $argv {  //Encapsulate command parameters
   append cmd "$[string length $a]\r\n$a\r\n"
}
::redis::redis_write $fd $cmd  //Send test commands to the test Redis instance
…}

With this, the test client can send the commands in the test case to the test instance, and determine whether the test is executed correctly based on the returned result.

I have drawn a diagram showing the interaction between the test server, test client, and test cases, as well as their main responsibilities in the test framework. You can take a holistic look at it.

## Conclusion

In today's lesson, we learned about Redis's unit testing framework. This testing framework is developed using the Tcl language, so before studying this framework, we need to first grasp some basic knowledge of Tcl language development. Since Tcl language itself has relatively simple data types, learning Tcl language mainly involves understanding the numerous keywords and commands it uses. This is also the content you can focus on learning next.

In the implementation of the unit testing framework, there are three main roles involved: **test server, test client, and test cases**. Their relationship is as follows:

- After the test server starts, it is responsible for starting the test client and interacting with the test client. It sends test cases to the test client through the "run test case name" message.
- After the test client establishes a connection with the test server, it sends a "ready" message to the server. Upon receiving the "run test case name" message sent by the server, the client introduces and executes the corresponding test script using the execute_tests function.
- The test script starts the Redis instance for testing using the start_server function, and then uses the r function provided by the test client to send test commands to the test instance. The r function actually calls the ::redis:: **dispatch** raw__function to complete the command sending.

Finally, I would like to remind you again that if you want to further study and master the Redis unit testing framework, you must have a clear understanding of the relationship between the test server, test client, and test cases summarized earlier. This way, you can understand how the entire testing process is conducted. In addition, since Tcl language development is relatively simple, after learning the Redis unit testing framework, you can also use it as a reference to implement your own testing framework.
## One question per lesson

In the Redis source code, there is a small testing framework for SDS. Do you know which code file this testing framework is in?