32 Testing From Development to Refactoring Testing #

Hello, I am Ishikawa.

In software engineering, there are many concepts, among which agile delivery is talked about the most. Although people have always emphasized agility and fast delivery, they often overlook the fact that agility does not mean sacrificing quality. The problem is that while emphasizing development speed, delivery quality is often overlooked. As a result, the concept of Test-Driven Development (TDD) emerged. Although Behavior-Driven Development (BDD) later appeared, TDD remains the most mature development model for developers when paired with agility.

Today, let’s take a look at test-driven development.

Red/Green/Refactor #

There is an important concept in Test-Driven Development (TDD) known as the “red/green/refactor” cycle. In this cycle, there are three steps:

• The first step is to create a failing test. Since we haven’t developed the relevant functionality yet, the test will definitely fail. This is the “red” part.
• The second step is to write the code that just passes the test. This is the “green” part.
• The third step is to refactor. In this step, we examine the code to see if there are any parts that can be optimized.

At first glance, you might think this is counterintuitive. TDD is the opposite of traditional development, where traditionally we should do step 3 first, design the “elegant” code structure; then do step 2, write the code; and finally do step 1, test the code developed based on our design.

So why do we do the opposite in TDD? The reason is that our test cases are not imagined out of thin air. They are created based on our user stories and acceptance criteria. The purpose is to clarify the development goals and desired results before starting step 1, and then step 2 is to write the code that fulfills our goals, which is the process of achieving the goals step by step. At this point, if our code is written well, it will naturally pass the test, thus avoiding the accumulation of numerous problems before discovering issues with the previously written code.

But why is refactoring step 3? Because for most of our projects, especially business-driven projects, time is money and efficiency is life. Being able to develop runnable code on time is more important than developing elegant code. Runnable code directly benefits users, while elegant code is often more of a concern for programmers.

You may be worried that this development pattern, which pursues results repeatedly, will lead to long-term overemphasis on results and impact code quality, resulting in long-term technical debt. Don’t worry, this is exactly the problem that the “red/green/refactor” cycle aims to solve. Refactoring is not done after a burst of technical debt, but is a step in each development cycle. In other words, it still exists and continues to iterate. This avoids excessive initial design while ensuring continuous optimization and iteration.

The purpose of refactoring is usually to optimize our software design by adjusting the code structure and removing duplication, making our code easier to understand. So although its purpose is not to deliver the Minimum Viable Product (MVP) faster, it can make our code more understandable and easier to maintain. As we have mentioned before in functional programming, after all, the code we write is more for humans to read than for machines.

Testing TDD or Behavior-Driven Development (BDD) #

We know that besides TDD, there is another concept called BDD development. Essentially, the testing process for BDD and TDD is very similar, with the main difference being the perspective of the tests.

For TDD, test cases are written with code and are oriented towards programmers. On the other hand, BDD test cases are usually written by business personnel or users, so the cases can be in the language we normally use, such as Chinese or English, rather than code.

The core focus of TDD is on testing the smallest unit, which is the unit test. It is then followed by integration tests and end-to-end tests. Because BDD testing is already at the level of behavior, such tests usually need to be run on an end-to-end basis. This means that if we want to use the red-green-refactor cycle in BDD, it will also involve smaller TDD unit tests within it.

This leads to the next question: the red-green-refactor cycle can be nested in a loop. For example, as shown in the figure below, we have written a failing test in the top left corner, represented by the red status. From this state, there are three possibilities. First, we can write the next failing test case based on the first failing test. Second, we can write the relevant program to pass the test based on this failing test case. Third, if we cannot pass the current test, we need to nest another level of loops, creating a new red-green-refactor cycle inside it to continue creating new failing tests.

For a test in the green state, there can also be two scenarios. The first scenario is refactoring, and the second scenario is completing the call, leaving the loop, meaning the test has passed and there is no code that needs further refactoring. Finally, for the refactoring state in blue, we can continue refactoring based on the test results or complete the code testing and leave the loop.

Once all tests are completed, all cycles will leave the loop. If we are in an inner loop, it means we enter the green stage of the outer loop. If we are in the outer loop and cannot come up with more tests to write, then the overall testing is completed.

It is worth noting in this process that a new red test is only created after the green test has passed and there is consideration for refactoring . If there is no need for refactoring, it means there is no more work to do. For example, if there was just an optimization done recently and there is no need for further refactoring, we can exit the loop directly.

In nested testing, we refer to the outer layer of tests as high-level tests and the inner layer as low-level tests . If the above theory still seems abstract, we can look at a specific example of a nested TDD cycle. Suppose we have a user login module, in which we can have the following two layers of tests.

1. The failing test case for the outer loop can be: “The customer can log in”. * The failing test case for the inner loop can be: “The login route /login returns a 200 response”. * To pass the low-level test, we need to write and test the code for routing until the test state turns green. * After passing the low-level test and turning the state to green, we can start refactoring the code in the inner loop. * At this point, the outer loop test may still fail, and we find that there is a missing inner loop test, so we write a new failing test for the inner loop: “The form post route /login-post should redirect to /personal-center if authentication via SMS and social media login is successful”. * To pass the new test, we write the code to handle the successful authentication via SMS/social media and return to the user’s personal homepage. * This new inner loop test has also passed, so now we can refactor the code written in the inner loop.
2. Now, both the low-level tests from the inner loop and the high-level tests from the outer loop have passed.
3. When all the test cases in the outer loop test turn green, we can then refactor the code in the outer loop.

Extension: Besides unit tests, what other types of tests are there? #

Previously, we mentioned that the most essential part of TDD is unit testing. Because at this stage, we mainly focus on the details of the code and the implementation of the application itself, so both development and testing may be done locally. So what can be considered as a unit?

In many languages, a unit refers to a class or function. In JavaScript, a unit can be both a class or function and a module or package. No matter how we define unit tests, the emphasis is on the behavior of the input and output of each unit, as well as the objects created during the process. In this process, we may also have dependencies on upstream and downstream systems, but these dependencies are usually simulated by using mocks and stubs. Through this approach, the testing between systems can be postponed.

After all the self-tests pass on the front end, we will continue with integration testing. Integration testing usually involves testing between systems that are directly related to our own system. At this time, the upstream and downstream systems may still be part of our own system, especially in the concept of the front end. Multiple front ends may share content provided by a backend server, and the relevant content services may be provided through BFF (backend for front-end).

Only after the tests at this stage pass, will we move on to end-to-end testing with external systems. In end-to-end testing, we may need to perform integration testing with more layers of systems. For example, in an e-commerce feature, when placing an order on the front end, it may first call the payment system’s interface to complete the payment, send the e-commerce order request, receive the result after successful payment, and then send a notification to the e-commerce system for shipping. The e-commerce system will further notify downstream systems to deliver the goods, and then return the delivery notification to the front end. Such a complete chain of functionality requires support from end-to-end testing. For this type of testing, the application code should be integrated with the code of other systems. Therefore, it is important to avoid using mocks or stubs to simulate the interaction between the front-end and back-end systems, and instead emphasize that the interfaces between the front-end and back-end systems are actually working, based on integration between systems.

Summary #

In many books or articles about TDD testing, they usually tend to be conceptual, hoping that through today’s learning, you can have a more concrete understanding of it. In the next lecture, we will discuss its implementation in a more practical way.

There are many testing tools developed around JavaScript. Moreover, these tools often exist in a modular way, which means one type of testing can be done with Tool A, and another type of testing can be done with Tool B. In the next lecture, we will take a look at Jest, which has relatively broader support for different types of testing compared to other testing frameworks. This avoids the need for us to switch tools between different testing scenarios.

In addition to functional testing, there are also non-functional testing. Non-functional testing includes performance testing, security testing, and accessibility testing, among others. We will further explore these topics in the third lecture of the “Testing Trilogy”.

Reflection Questions #

Today we learned about Test-Driven Development from the perspective of red-green-refactor. Here we primarily focus on the depth (nesting) of the tests. In addition to that, we may also need to pay attention to coverage and complexity (number of circles). Could you share whether you have the habit of Test-Driven Development in your development work? And what is the typical coverage rate of your tests?

Feel free to share your experiences, exchange learning insights, or ask questions in the comments section. If you found it helpful, you’re also welcome to share today’s content with more friends. See you in the next class!