09 Can Your Work Be Measured by Numbers? #
Hello, I am Zheng Ye.
Today’s sharing starts with daily work. Please think back to what is the first thing you do after arriving at work every day? And then guess what my answer is. You may not guess it, but my first task after arriving at the company every day is to look at numbers.
I now work for a company that deals with digital assets, and we provide a 24-hour running service. From the first day of joining this company, the people at the company have constantly instilled in me an important concept - to look at numbers. In front of my seat, there is a huge monitor displaying various constantly changing curves, bar charts, and numbers, which reflect various system operation indicators.
We look at these indicators every day to discover any problems with the online systems. Once certain indicators show abnormalities that I cannot understand, investigations are conducted.
You may wonder, aren’t we discussing “beginning with the end in mind”? How did it become a discussion about monitoring? Don’t worry, we are still discussing “beginning with the end in mind” because numbers are the best way to interpret the “end”.
We have discussed various “ends” before, but the “ends” defined by language tend to be somewhat vague and prone to misunderstandings. Numbers, on the other hand, are clear-cut “ends”. For example, if the test coverage requirement is 100%, even if you achieve 99.9%, it is still unacceptable, there is nothing more to say, it simply does not meet the standard.
Another example is when we talked about lean startups earlier, we mentioned an important feedback loop: build-measure-learn. You will find that in this loop, building (build) is controllable, learning (learn) can only be achieved after receiving feedback. So, the most important question we need to answer here is measuring (measure). And the best criterion for testing here is, of course, numbers.
Perhaps you would say that we are all familiar with numbers, so why talk about it? However, in my opinion, you may not be accustomed to using numbers.
Familiar and Strange Numbers #
From an evolutionary perspective, people rely more on intuition when they do things. Numbers are something that humans created after running on the African savannah for a long time. There is such a story at the beginning of the famous popular science book “From One to Infinity”:
Two Hungarian aristocrats decided to play a counting game to see who could say the larger number. - One aristocrat said, “Alright, you go first!” - The other aristocrat, after thinking hard for several minutes, said a number: “3”. - Now it’s the first aristocrat’s turn to think hard. After thinking for a quarter of an hour, he said, “Alright, you win!”
This story sounds absurd, but some African explorers have confirmed that in certain primitive tribes, there are no number words larger than 3. If you ask them how many children they have, and the number is greater than 3, they will answer “many”.
Although the Chinese nation is a nation that values education, we now all recognize that mathematics is an important fundamental knowledge. However, we still tend to have a rough understanding because there are not many occasions in daily life where we need to pay much attention to numbers, apart from buying things and receiving wages.
As the wheels of history keep rolling forward, the complexity faced by today’s society has far exceeded the level at which things can be done solely by intuition.
Some people say that they can do things well based on intuition, but this is actually a misconception because the so-called intuition is usually a kind of insight, which largely depends on a person’s long-term accumulation and knowledge in a certain field, and this is actually a kind of big data in a sense.
We all say that human beings are about to enter the era of intelligence. The main reason for saying this is that artificial intelligence technology is constantly advancing. As a technology that has been around since the 1950s, artificial intelligence has only made great strides in recent years, mainly due to the development of infrastructure.
In the field of artificial intelligence, the statistics-based approach has long been proposed in academia, but due to the limitations of technology at the time, people’s data collection and storage capacities were limited, and the concept of “big” data at that time was completely different from today’s big data.
It was not until the Internet age that, with the increase in data processing volume and the continuous expansion of infrastructure, people began to collect more and more data, which in turn created today’s big data.
AI algorithms that were originally unable to work effectively due to a lack of sufficient data support suddenly found a stage to perform in today’s world, transforming from a marginal role to the center stage.
When it comes to artificial intelligence today, people mainly talk about three things: algorithms, computing power, and data. Algorithms are almost universally shared in the industry, and with the popularization of cloud computing, computing power is no longer a scarce resource. Therefore, data has become the “commodity” that each company strives for. So, data collection is an industry consensus in the broad sense.
However, as a group of people who understand the value of data the most in this world, we programmers are only trying to use data to constantly improve the lives of others, and we seldom think about improving our own work in daily life.
We are still more accustomed to discussing things based on intuition. For example, adding this feature “may” increase users, and making this adjustment “should” reduce the stress on the system.
In some simple situations, or in situations where information is symmetric and there is little difference in knowledge background, such discussions are easy to get consensus. However, when things become complex to a certain extent, it is difficult to convince people simply based on feelings.
Therefore, it often happens in our work that one person says they think something is useful, while another person says they think it is not. After a few rounds of “I think”, the two sides start to engage in a debate. Neither side can persuade the other.
If we switch to discussing using numbers, the effect will be better. Once, in order to improve user experience, we were preparing to make a major revision to the homepage. The product team wanted to add a lot of content to the homepage, while the development team believed that too much content would slow down the loading speed of the homepage and consequently lead to a decrease in user experience.
Just as this discussion was about to enter the stage of “empty talk”, we found a metric: homepage loading speed. As long as the loading speed was ensured, the product team could make adjustments according to their own understanding. As a result, an irretrievable discussion turned into a discussion under certain constraints, and neither side could let their thinking wander. The discussion could then continue to progress.
If you agree with the value of data itself and combine it with the concept of “starting with the end in mind,” then before starting to do something, we should think about how to measure it. Whether it is discussing product features or functionality development, it is easy to “talk nonsense”. But when it comes to numbers, people will think about it more. This is a constraint on each other.
Starting from Numbers #
In the previous content, we have been emphasizing the importance of measurement indicators and numbers. Next, I will share several examples of how I have used numbers in my actual work, to show you how thinking about problems with numbers can broaden your thinking.
First, let’s talk about making technical decisions based on numbers. Once, we wanted to make a technical improvement and add some caching to the system to reduce the pressure on the database. We designed two technical solutions together. If the query was specific, we would simply add caching to certain methods; if the query was diverse, we would use a middleware and its query scheme.
What exactly is the current situation of the system? We found that we couldn’t answer this question immediately. So, we decided to add some statistical indicators to the system to let the data provide us with answers. Then, based on the situation reflected by the data, we made specific decisions.
Next is a case of preparing for system deployment. At that time, we were planning a major system upgrade that would have a significant impact because it involved a critical module that would affect upstream and downstream systems.
No one could be sure which module would encounter problems during the deployment. Therefore, we designed a brand new data dashboard that displayed the core indicators of several related modules. Our task was to observe the changes in these indicators while the system was being deployed.
Fortunately, the impact of this deployment was minimal, with the indicators remaining stable throughout the process. The confidence of the team was derived from these well-prepared indicators.
Moving on, let’s take a look at an example where a problem was discovered through numbers. Due to various historical reasons, our key indicator panel had several indicators that represented similar things.
For example, the processing capacity of a certain module had one indicator measured internally within the module, while another indicator was measured by upstream and downstream systems of the module. In most cases, their performance aligned. However, one day, they suddenly displayed a significant difference. The internal measurement remained good, while the external measurement experienced significant delays.
As a result, we began questioning why this was happening. Through continuous investigation, we discovered that the module needed to periodically persist its internal state, and during that time period, it would stop reading data from the upstream. So, everything appeared normal internally, but there was a significant delay externally. Subsequently, we found a solution and resolved this issue.
Finally, let’s discuss an example from the industry. To my knowledge, some companies in the industry have started doing something called AIOps, which means using artificial intelligence to discover more operation and maintenance problems from data. Regardless of the approach, the goal is to discover problems from numbers and make systems more stable.
A colleague of mine made a very interesting point. He said that from a numerical perspective, a good system should be “like still water”.
I agree with this point because if there are fluctuations, especially significant ones, without a reasonable explanation, it indicates hidden risks in the system. Making the system stable is an important part of our work.
Returning to the beginning of this lecture, I mentioned that an important aspect of daily work is to look at numbers, which is essentially an attempt to discover problems from the trends revealed by numbers. Nowadays, the team has become accustomed to communicating with phrases like “show me the numbers,” which has reduced some internal disputes.
Summary #
With the advent of the intelligent age, human society is gradually realizing the importance of data. However, while we IT professionals serve others through data, we rarely bring digital thinking into our own work. This is also the root cause of many “empty to empty” conversations in our work.
Combining the thinking of “starting with the end in mind,” if we can design metrics to measure the effectiveness of our work from the beginning, then we can work more purposefully.
And if we are accustomed to thinking in numbers, we can let numbers help us in many ways. I have mentioned a few examples, such as: making technical decisions based on data, pre-defining system metrics, and discovering problems in the system, etc. I hope you can also bring digital thinking into your daily work.
If there is one thing you remember from today’s content, please remember this: Ask yourself, can my work be measured in numbers?
Finally, I would like to ask you to share, what are some scenarios in your work where you use digital methods to solve problems? Please feel free to write down your thoughts in the comments.
Thank you for reading. If you found this article helpful, please feel free to share it with your friends.