25 Problems Keep Arising in Development, What Should I Do? #

Hello, I’m Zheng Ye.

Most students who have watched “Saint Seiya” are deeply impressed by one of the sayings: Saints will not be defeated by the same move twice.

We all hope that our development skills can be as powerful as the Saints’, but reality often does not go as planned: the same online failures keep occurring, similar bugs repeatedly cause trouble in different places, and talented colleagues spend their lives “putting out fires” every day. Can’t we improve a little?

If the same problem keeps occurring during the development process, it means your team is not doing a good job of conducting retrospectives.

What is Retrospective? #

Retrospective, originally a Go term, refers to the process of replaying a game after it is played, and examining which moves were good, which were not, and where there could have been different or better moves.

This process of restoring and discussing the game is called retrospective.

Nowadays, the concept of retrospective has been applied to many aspects, such as stock market retrospective and corporate management retrospective. It has become an important tool for individuals and companies to continuously improve. One of the most famous examples is Liu Chuanzhi, the founder of Lenovo, who even included “retrospective” in Lenovo’s core values.

Why is retrospective so useful? In my opinion, one important reason is objectification.

As the saying goes, those who are in the midst of things are easily confused, but those who are on the sidelines see clearly. Using software development as an example, when solving a problem, our attention is mostly focused on solving the problem itself, and we rarely think about how the problem arose.

When you conduct a retrospective, you take a different perspective to think about the causes of the problem. At this point, you are no longer a participant, but an observer. You observe the process of the event as if someone else were doing it. You shift from a subjective perspective to an objective perspective.

Looking at the problem from someone else’s perspective, that is objectification.

In the field of software development, retrospective is also an important practice, used to address the recurring problems mentioned earlier, but presented in different ways.

Review Meeting #

A review meeting is a common practice for reflection. Regular reviews are a prerequisite for team self-improvement. How do we conduct review meetings? Let me share with you my usual approach.

As the organizer, I start by presenting a theme classification on the whiteboard. I usually divide it into three categories: “What we did well”, “What we did poorly”, and “Issues or suggestions”.

There are also different ways to classify the themes, such as using a starfish diagram, which is divided into five categories: “Continue doing”, “Start doing”, “Stop doing”, “Do more of”, and “Do less of”.

The classification method can be chosen based on the preferences of your team. I chose the three-category classification because it is simple and intuitive, requiring little further explanation for each category.

Then, I give the participants five minutes to write down facts on sticky notes based on the team’s performance during the development cycle, according to the categories. For example, you might think that delivering on time is something we did well, and having too many bugs is something we did poorly.

There are two key points here. One is to write facts, not feelings. Facts are objective and measurable, whereas feelings cannot be quantified. What you feel is good may feel terrible to someone else.

Also, only write one item per sticky note because later on, we will be categorizing them. Since everyone writes individually, there may be a lot of duplicated content, so we need to categorize them.

After five minutes, I call everyone to stick their notes on the whiteboard. Once all the notes are in place, I read them one by one.

This is done to let everyone know what others have written and to understand different people’s areas of focus. If there is any item that is unclear, I ask the author of that note to explain it so that everyone understands the issue consistently. While I read the notes, I also categorize them.

Once all the notes are categorized, they become the topics for subsequent discussions. The participants also have a better understanding of everyone’s areas of concern and the problems they see.

The parts where we did well are areas where we should encourage ourselves and continue doing so. However, since the main purpose of our review meeting is to improve and elevate ourselves, our focus is on addressing the areas where we did poorly and where issues arose.

Before starting a more targeted discussion, I ask everyone to vote on the most important topics from these categories. I usually give each person three votes to allocate to the topics they consider important. Each person needs to make choices among various content. If you deem a particular topic extremely important, you can allocate all your votes to that topic.

Based on the voting results, I establish an order for all the topics, and that becomes the sequence of our discussions. We cannot have unlimited meetings, so usually, only the most important topics will be discussed.

Whether it is individuals choosing topics they want to discuss or the team collectively selecting the topics for discussion, everyone should have the concept of “priority” in mind. Then, we will discuss the topics one by one according to the order.

When discussing a specific topic, we first focus on the current situation. I ask the person who wrote the feedback to briefly describe what they have observed. For example, a tester might say that there have been more bugs recently, with a 50% increase compared to the previous development cycle.

Then, I ask everyone to analyze the reasons for this situation. For example, someone might say that the workload has been heavy recently, and there hasn’t been enough time for testing.

Next, we try to find a solution and propose actionable items. For example, if the workload is heavy, we can ask the project manager to control the input of requirements more effectively and reduce unnecessary demands. If testing has been neglected, we can consider adding test coverage to the build script, so code submission is not allowed if the test coverage is insufficient.

Please note that all proposed actions should be verifiable, not vague or unmeasurable. For example, if the action item is for every programmer to be “more careful,” it is not achievable. “Carefulness” is subjective, and if you state that programmers are not careful, they might argue that they are. This will only lead to pointless disputes.

On the other hand, the proposed actions mentioned earlier are verifiable. Project managers controlling requirements can be measured by workload, just like we discussed in evaluating user stories. How can we measure workload control? We can look at the total story points for each phase of development and see if it has decreased compared to the previous phase. Test coverage is more straightforward; we can simply write it into the build script and disallow code submission if the test coverage is insufficient.

Great! We have listed actionable items. The next step is to find responsible individuals who will be accountable for these items.

For example, the project manager will be responsible for requirement control, and the technical lead will be responsible for adding test coverage to the build script. With responsible individuals assigned, we ensure that these tasks are not neglected. When we conduct the next review, we can ask the responsible individuals about the status of completing these tasks, using them as checklist items.

5 Whys #

Whether you conduct retrospective meetings or analyze problems in order to find the root cause, both are important steps.

If your team can immediately identify the root cause, that’s great. But if they cannot, it’s best to ask more “whys”. One common approach is the “5 Whys” technique. This practice was introduced by the founder of Toyota, Sakichi Toyoda, and later became well-known through the Toyota Production System.

Why should you ask more “whys”? Because with the initial question, you can only uncover surface-level causes. It is only by asking several more “whys” that you can potentially find the fundamental cause.

Let me give you an example. If a server frequently returns a 504 error, we can use the “5 Whys” technique to ask:

• Why does the 504 error occur? Because the server’s processing time is too long and exceeds the timeout.
• Why does it exceed the timeout? Because the server gets stuck when accessing Redis.
• Why does accessing Redis get stuck? Because another service that updates Redis deleted a large batch of data and then inserted it again, causing the server to block.
• Why did it delete and insert a large batch of data? Because the update algorithm was poorly designed.
• Why was a poorly designed algorithm allowed to go live? Because the design did not go through the proper review process.

By asking these questions, you can identify the root cause of the problem: the design not going through a review process. Once you find the cause, the solution naturally becomes apparent: critical algorithms must go through a review by relevant personnel.

Of course, this is just an example. Sometimes, this answer may not be sufficient to solve the problem, and we can continue asking more questions, such as why the review process wasn’t followed, and so on.

Therefore, the “5” in “5 Whys” is just a reference, not a strict target.

The “5 Whys” technique is a simple and easy-to-use tool. Just by hearing its name, you may already know how to use it. One thing to note is that the questions should follow a single line of inquiry and should not be five unrelated questions.

Whether it’s a retrospective meeting or the “5 Whys” technique, the most important aspect to consider is not to use these methods to blame someone. Our goal is to solve problems and continuously improve, rather than launching emotional criticism against individuals.

Summary Moment #

In software development, many problems tend to recur, and many development teams can get caught up in an endless cycle of putting out fires. One effective way to address this is through retrospective analysis.

Retrospective analysis, or “retros,” involves recounting the process, conducting discussions and analysis, and finding ways for self-improvement. This approach allows us to have an objective perspective and objectively evaluate everything that has happened. This method has been widely applied in many fields, such as stock market and enterprise management.

In software development, there are also practices that involve retrospectives. I will give you a detailed introduction to the “retrospective meeting” format.

Regardless of the approach, analyzing problems and identifying root causes is a crucial step. “The 5 Whys” is a commonly used technique to uncover root causes.

If there’s only one thing you remember from today’s content, please remember this: regularly conduct retrospectives, identify problem root causes, and constantly strive for improvement.

Lastly, I’d like to ask you to share how your team addresses these recurring problems. Feel free to write about your approach in the comments section.

Thank you for reading, and if you found this article helpful, please consider sharing it with your friends.