23 Basic Rules and Processes of Testing - Part 1 #

Hello, I’m Lin Hao, and today I will be sharing the topic of basic rules and processes of testing (part 1).

You’re doing great, you have completed the largest module of this column! It covers all built-in data types in Go language, as well as those distinctive processes and statements.

You are now fully capable of independently writing various Go programs. If you forget anything, you can review the previous articles.

In the days to come, I will guide you in learning additional knowledge that is essential for advanced Go language programming, such as Go program testing, program monitoring, and the correct usage of various commonly used code packages in the Go language standard library.

From the last century to today, programmers, especially Chinese programmers, have been tirelessly writing programs, even staying up all night (like myself).

Because this is a unique way for ordinary people like us to self-train, change our lives, and even change the world. However, when it comes to writing programs for testing purposes, we often shy away from it. Why is that?

In my personal opinion, from human nature, we tend to deny “self-denial.” We don’t want to see bugs (program errors or defects) in the programs we write, especially when we have put our hearts into writing and confidently delivered the program.

However, what I want to say is that whether a person can progress and how fast they can progress relies precisely on self-denial, including the depth and frequency of self-denial. This is exactly what the phrase “out with the old, in with the new” means.

For programs and software, it is actually crucial to discover and correct problems as early as possible. In the context of this interconnected network, the programs, tools, or software products we create can spread faster and farther. However, at the same time, their errors and defects can spread as well, and may affect thousands or even more users in a short period of time.

You might say, “In the open source model, this is an advantage. I want more people to help me discover and even fix errors. We can collaborate and maintain the program together.” But these are actually two different things. Collaborators often come from early or core users, but it cannot be assumed that program users will become collaborators.

When many users begin to complain about a program, it may indicate that your reputation is about to collapse. You will find, or someday you will find, that the more people care about and love a program, the more comprehensive the testing (especially automated testing) is and the more standardized the testing process becomes.

Even if you want to gather more firewood for a bigger flame, you need to make others like your program first. Moreover, for excellent programs and software, testing is inevitably a highly valued aspect. So, build a fortress for your program as soon as possible with testing!

There are also many types of testing for programs or software, such as unit testing, API testing, integration testing, gray box testing, and so on. In this module, I will mainly focus on explaining unit testing.

Introduction: Basic knowledge of testing in Go programs #

Let’s talk about unit testing, also known as developer testing. As the name suggests, this is one of the self-checking tasks that programmers should do.

The creators of the Go language have always attached great importance to program testing and have provided developers with rich APIs and tools. With these APIs and tools, we can create test source code files and write test cases for program entities in command source code files and library source code files.

In Go, a test case is often represented by one or more test functions, although in most cases, one test function is sufficient for each test case. Test functions are often used to describe and ensure the functionality of a program entity, such as what kind of input produces what kind of output under normal circumstances, or when the functionality reports errors or behaves abnormally under certain conditions, and so on.

In Go, we can write three types of tests: functional tests (test), benchmark tests (benchmark, also known as performance tests), and example tests (example).

From the names, you should be able to guess the purpose of the first two types of tests. Strictly speaking, an example test is also a type of functional test, but it pays more attention to the content printed by the program.

In general, a test source code file only tests a command source code file or library source code file (referred to as the target source code file), so we always (and should) put them in the same code package.

The main name of a test source code file should start with the main name of the target source code file and must end with “_test”. For example, if the target source code file is named “demo52.go”, the name of the test source code file for it should be “demo52_test.go”.

Each test source code file must contain at least one test function. From a syntactic perspective, each test source code file can include test functions for any type of test, even if all three types of test functions are included. I usually do it this way, as long as I control the grouping and number of test functions.

We can divide these test functions into different logical groups based on the different program entities they target. We can also use comments and helper variables or functions to make divisions. Additionally, we can arrange the order of test functions in the test source code file based on the order of program entities in the target source code file.

Furthermore, Go language also has explicit rules for the names and signatures of test functions. Do you know what these rules are?

So, today’s question is: What are the rules for the names and signatures of test functions in Go?

Here are the typical answers to this question:

• For functional test functions, their names must start with Test and the parameter list should only contain one declaration of type *testing.T.
• For benchmark test functions, their names must start with Benchmark and the only parameter must be of type *testing.B.
• For example test functions, their names must start with Example, but there are no specific requirements for the parameter list of the function.

Problem Analysis #

There are generally two purposes for asking this question.

• The first purpose is to examine the basic rules of testing Go programs. If you frequently write test source code files, then this question should be easy to answer.

• The second purpose is to serve as an introduction to the second question: What is the main testing process performed by the go test command? However, I won’t ask you that here; I will just give you the answer.

First and foremost, we need to remember that only if the name of the test source code file is correct and the names and signatures of the test functions are correct, will the test code be executed when we run the go test command.

When the go test command starts running, it will first do some preparation work, such as determining the internal commands needed, checking the validity of the specified code package or source code file, and determining if the provided flags are valid, etc.

After the preparation work is successfully completed, the go test command will build, execute the required test functions in each tested code package one by one, clean up temporary files, and print the test results. This is the main testing process under normal circumstances.

Please note the word “sequentially” in the above description. For each tested code package, the go test command executes each step of the testing process in a serialized manner.

However, in order to speed up the testing process, it usually performs functional tests on multiple tested code packages concurrently. However, when printing the test results, it will execute them one by one according to the order we specify, giving us the impression that it is executing the testing process in a completely serialized manner.

On the other hand, since concurrent testing can cause deviations in performance test results, performance tests are generally performed sequentially. More specifically, performance tests will only start after all build steps are completed in the go test command.

In addition, the performance test of the next code package will always wait until the printing of the performance test results of the previous code package is completed before starting, and the execution of performance test functions will also be sequential.

Once the specific testing process of Go programs is clear, some of our doubts will naturally have answers. For example, why the test function named testIntroduce did not execute, and why even simple performance tests take longer to execute compared to functional tests, etc.

Summary #

From the beginning of this article, I have been trying to explain to you the importance of program testing. In at least half of the companies I have experienced, program testing is not taken seriously, or there is no time or resources dedicated to it.

Especially in small and medium-sized companies, they often rely entirely on software quality assurance teams or even real users to test for them. In these cases, the cycle of discovering, reporting, and fixing software errors or defects can be long and costly, and may also have a negative impact.

The Go language is a programming language that places great emphasis on program testing. It not only comes with the testing package, but also has the go test command dedicated to program testing. To truly make good use of a tool, we need to first understand its core logic. So, my first question to you today is about the basic rules and main process of the go test command. After learning about these, you may have a deeper understanding of Go program testing.

Thought-provoking questions #

Apart from the ones mentioned in this article, do you know of or have you used any methods of the testing.T type and testing.B type? What are they used for? You can leave me a message and we can discuss together.

Thank you for listening, see you next time.

Click here to view the detailed code accompanying the Go language column article.