12 Is Testing a Programmer’s Responsibility? #

Hello, I am Zheng Ye.

In the “Task Decomposition” module, I am going to start with a topic that really helps me understand task decomposition, which is “testing”.

This is a topic that programmers both love and hate. They love testing because it ensures the quality of the project, but they hate it because writing tests can be difficult. In reality, many people struggle with writing tests mainly because they don’t understand task decomposition.

In the previous module, we mentioned some best practices, but they were all explained from the perspective of “starting with the end in mind”. This time, I am going to change the approach and use five articles to fully explain “developer testing” so that you and I can rediscover this topic that you may have overlooked.

Are you ready? Let’s start with the topic that confuses many people: Should programmers write tests?

Who Should Do Testing? #

As a programmer, you certainly know why testing is necessary. Since we are developing software, we need to ensure its correctness as much as possible. After all, having basic professional ethics is essential.

But who should be responsible for testing? This is an interesting topic. Many people intuitively think that testing should be the job of testers, so why even ask?

It is true that testing should be done by testers, but is testing solely the responsibility of testers?

In fact, as a programmer, you have probably already done a lot of testing work yourself. For example, before submitting your code, you definitely run it to ensure that the basic functionality is correct. This is the most basic form of testing. However, you may not consider it as testing most of the time, so in your intuition, testing is the job of testers.

But I still want to emphasize that testing should be part of a programmer’s work. Why do I say that?

Let’s think about what testers can test. Yes, they can only test the functionality and features of the system from the outside. However, a software system is composed of many internal modules. Testers can only ensure correctness from the outside, and their scope of effectiveness is limited.

Let’s use an analogy. Imagine you are building a machine and each component does not guarantee correctness, but the final result must be correct. This is an absurd requirement, but it happens in the software development process.

In software development, there is an important concept: the cost of software changes, which increases gradually over time and development stages. This means that we need to discover and fix problems as early as possible, so that the cost involved is the lowest.

In the previous section, we talked about “starting with the end in mind,” which emphasizes the importance of early problem discovery. If a problem can be solved at the requirement stage, it should not be left until the development stage. Similarly, if a problem can be solved during development, it should not be left until the testing stage.

You can think about it. Is it easier for you to discover and fix errors in your code, or to test and locate problems after they have been reported?

The ideal situation is to ensure quality throughout the entire software development process, considering testing in every aspect from the beginning of the requirements. When each role delivers their work, an important question should be asked: How do you ensure the quality of the deliverables?

The requirements team should define acceptance criteria, and the development team should deliver their developer tests. This is an important concept derived from lean principles: build quality in.

Therefore, for every programmer, only by writing good code and tests during the development stage can they claim to deliver high-quality code.

Automated Testing #

Unlike traditional testers who only verify through manual means, programmers have a natural advantage when it comes to testing: the ability to write code. This advantage allows us to automate testing.

In the early days of testing code, the simplest approach was to create a separate program entry point. I have also done this when I first entered the workplace, as it seemed like an intuitive approach. However, since programmers have a need to write tests and this need keeps recurring, better automation solutions emerged. Thus, testing frameworks were born.

The earliest testing framework originated from Smalltalk. Smalltalk is an early object-oriented programming language with many enthusiasts. Many popular programming concepts today come from Smalltalk, and testing frameworks are one of them.

The widespread popularity of testing frameworks can be attributed to Kent Beck and Erich Gamma. Kent Beck is the founder of Extreme Programming and is well-known in the field of software engineering. Erich Gamma, on the other hand, is the author of the famous book “Design Patterns,” and has made significant contributions to projects such as Visual Studio Code that many people are familiar with.

Once, the two of them were flying from Zurich to Atlanta to attend the OOPLSA (Object-Oriented Programming, Systems, Languages & Applications) conference. During the flight, they paired up and wrote JUnit together. From this name, it is not difficult to see that their goal was to create a unit testing framework.

By the way, if you know that Kent Beck is an avid fan of Smalltalk and has previously written the SUnit testing framework, it is not difficult to understand how these two individuals were able to accomplish such a feat during a single flight.

After JUnit, the concept of testing frameworks gradually gained popularity. In today’s “world of programming,” testing frameworks have become an industry standard. Every programming language has its own testing framework, and some even include it in their standard libraries. The industry also uses the term XUnit to collectively refer to these testing frameworks.

The greatest value of this type of testing framework is that it introduces automated testing as a best practice into the development process, allowing testing actions to be standardized and fixed through standardized means.

Testing Models: Egg Roll and Pyramid #

In the previous discussion, we categorized testing into manual testing and automated testing. Even if we focus only on automated testing, we can still divide it into different levels: unit testing that focuses on the smallest program modules, integration testing that combines multiple modules together, and system testing that combines the entire system.

Some people like to include acceptance testing in this categorization as well. For the sake of simplicity, let’s temporarily ignore acceptance testing.

The question that follows is how many different levels of tests should we write? In theory, more is better, but in reality, everything comes at a cost, so people have to make trade-offs. Depending on the proportion of different tests, different testing models are formed.

One intuitive approach is that since higher-level tests have broader coverage, we should write more higher-level tests, such as system testing.

Of course, there are some scenarios where higher-level tests are not easy to cover, so there should also be some lower-level tests, such as unit testing. In this case, lower-level tests only serve as supplements to higher-level tests, and the main focus is on higher-level tests. This forms a testing model called the “ice cream egg roll”.

Not many people have heard of the ice cream egg roll testing model. It is a time-consuming and labor-intensive model. The preparation for higher-level tests is just too cumbersome.

The reason why I mention it here is because although many people haven’t heard of this concept, quite a few testing teams actually adopt this model, which is the reason why many teams find testing troublesome without realizing why.

Next, let’s talk about another testing model, which is the industry’s best practice: the testing pyramid.

Mike Cohn introduced the testing pyramid in his book Succeeding with Agile, but most people learn about it through Martin Fowler’s article.

From the image, we can see that it is almost the reverse of the ice cream egg roll. The focus of the testing pyramid is that the lower the level of the test, the more we should write.

To understand why the testing pyramid has become the industry’s best practice, we need to understand the differences between tests at different levels. The lower the level of the test, the less related content it involves, while higher-level tests have broader coverage.

For example, unit tests have only one unit as their focus and nothing else. So, as long as one unit is written well, the test will pass. On the other hand, integration testing requires putting several units together to test, and the prerequisite for passing the test is that all these units have been written well. This cycle is obviously longer than that of unit testing. System testing requires connecting all modules of the entire system and preparing various data to pass.

The theme of this module is “task decomposition,” and I must emphasize one point: small tasks have shorter feedback cycles, while big tasks have longer feedback cycles. Small tasks are easier to do well, while big tasks are much more difficult. Therefore, based on this criterion, it is easier to write lower-level tests.

In addition, because it involves multiple modules, any adjustments made to one module may break the higher-level tests, so higher-level tests are usually relatively fragile.

Moreover, in actual work, some higher-level tests may involve external systems, which further increases complexity.

People instinctively tend to do less complicated things, so people certainly won’t tend to write more higher-level tests, and as a result, the amount of testing at the higher level will not be too high, and the test coverage will not be very high no matter what. Moreover, once a test fails, because there are too many things involved, it is also very difficult to locate the problem.

On the other hand, making lower-level tests the main focus and writing more of them is possible because they involve fewer things and are easier to write, which allows the team to have more tests and makes it easier to discover problems.

Therefore, although the ice cream egg roll is more intuitive, the testing pyramid is the industry’s best practice.

When the Test Pyramid Meets Continuous Integration #

The test pyramid serves as a foundation for an important practice called continuous integration. When the number of tests reaches a certain scale, the time it takes to run the tests can become quite lengthy, making it impossible to run all the tests at once in the local environment. Generally, we choose to run all unit tests and integration tests locally, while executing system tests on the continuous integration server.

At this point, the quantity of bottom-level tests becomes crucial. According to the test pyramid model, there will be a large number of bottom-level tests that can cover the major scenarios. However, according to the ice cream cone model, the quantity of bottom-level tests is limited.

As a safety net for code submissions, the quantity of tests determines how early we receive feedback. Therefore, the test pyramid model naturally cooperates well with continuous integration.

It is important to note that tests written using a unit testing framework are not necessarily unit tests. Many people use unit testing frameworks to write integration tests or system tests. A unit testing framework is simply an automation testing tool and not intended for defining the type of testing.

In actual work, there are various ways to differentiate between different types of tests. For example, placing different tests in separate directories or giving different types of tests a unified naming convention.

The main purpose of distinguishing different types of tests is to run different types of tests in different scenarios. As mentioned earlier, a common approach is to run unit tests and integration tests locally, while running system tests on the continuous integration server.

Summary #

Testing is an important part of software development, and it should be the responsibility of everyone on the development team, not just the testers. The cost of making changes to software increases over time and during the development stages. Problems that can be resolved early on should not be postponed to the next stage.

When it comes to testing, programmers have a natural advantage because they know how to write code. Therefore, automated testing is a strength that programmers possess. Writing tests should be an integral part of a programmer’s work.

As people’s understanding of testing deepens, various types of testing have emerged, and testing has been categorized into different types: unit testing, integration testing, system testing, and so on. The lower the level of testing, the lower the cost and the faster the execution. On the other hand, the higher the level of testing, the higher the cost and the slower the execution.

Human time and energy are limited, so people have started to think about how to combine different types of testing. The best practice in this regard is called the testing pyramid, which emphasizes that more tests should be written at the lower levels. Only by following the testing pyramid approach can continuous integration work effectively.

If there’s one thing you should remember from today’s content, it’s this: write more unit tests.

Finally, I’d like to invite you to share the difficulties your team has encountered in writing tests. Feel free to leave your thoughts in the comments section.

Thank you for reading, and if you found this article helpful, please feel free to share it with your friends.