Concluding Words We Have Just Started on the Road of Source Code Learning #
Hello, I’m Hou Xi. Today marks the end of the “Interpretation of Kafka Core Source Code” course, and it’s time for me to say goodbye to you.
When I took on this task, I remember the editor telling me that this course would be GeekTime’s first column course on source code. I felt proud and apprehensive at the same time.
I’m proud because I can share my experience of studying Kafka source code with you as the author of the first source code column course. I’m apprehensive because source code is not easy to explain. If the explanation is too coarse, it will be superficial and not profound enough. If the granularity is too fine, it is easy to get caught up in a detailed line-by-line code walk-through. Striking a balance between the two requires careful consideration.
Nevertheless, I am still willing to give it a try.
Because I have always felt that the current programming education doesn’t pay enough attention to reading source code. Schools and various training institutions have always focused on imparting programming skills but have neglected to teach the methods of writing source code for popular frameworks in the industry. However, this is crucial. If you haven’t read a large amount of high-quality code, it is impossible for you to write excellent code.
Just as American writer Stephen King said, “If you don’t have time to read, you don’t have the time (or the tools) to write. Simple as that.” Although this quote is about writing, it is equally applicable to writing code. Learning from the source code written by people with more experience than yourself will directly or indirectly influence your coding skills.
In the [introduction], I mentioned three benefits of reading source code. However, in fact, for a programmer who writes code every day, the ultimate goal of reading source code is to be able to write better code. Over the years, my experience of reading Kafka source code has made me deeply aware of this. Therefore, I really want to organize my reading experience and share it with you, helping you read source code more scientifically and efficiently, and ultimately write better code.
Since the course preparation began in December 2019 until today, it has taken nearly eight months. I have carefully selected and explained the core source code of the main components of the Kafka broker. Overall, I think it has been a success. I learned a lot from writing the source code analysis of many modules. Therefore, I think that if you have studied the previous content seriously, you should have gained something as well.
Of course, no matter how much you have learned at the moment, the knowledge that has been updated in the past will be left behind. They are waiting for you to consult, digest, and use them to solve current and future problems. At this moment, we have to say goodbye to them for now. I would like to talk to you about the future, the future of learning source code.
After the course is over, how should you continue your Kafka source code reading plan? Based on my experience, I would like to share some suggestions with you.
Firstly, I suggest that you study the code of the Kafka Clients component.
You may ask, shouldn’t we read the code of other components of the Kafka broker first? Why should we read the code of the Clients component? In reality, the Client project contains not only the source code of the client but also various basic codes that support server-side components, such as communication channels and message collections, and so on. Understanding their implementation can enhance your overall understanding of the Kafka code framework. Reading and understanding the code in the Clients and Core projects is a necessary step to master the implementation principles of Kafka.
Secondly, go through the test case code related to the components mentioned in the course, and it’s best to set breakpoints and debug while running. Many times, it is difficult to understand how a component actually works just by looking at the implementation source code. The test cases can lead us to explore it in detail.
Let me give you a real example. When I was studying the Log Cleaner source code, I found it very challenging. I read the LogCleaner class many times but still couldn’t grasp the essence. Later, I suddenly thought, isn’t there a LogCleanerTest? So I opened it and studied each test××× method, thinking carefully, and finally understood the actual working principle of this component. Therefore, I want to remind you specifically not to underestimate the power of test case code. Finally, you must take the time to read the code of the Log Cleaner component.
It is mainly used for completing log cleaning with key messages, and it does not appear much in other situations. That’s why I didn’t specifically cover it in the course. However, it is actually a very important component on the broker side.
The well-known offset topic relies on this mechanism to regularly delete expired offset messages and expired registration messages. In fact, clearing expired or duplicate data is a crucial issue for any data storage system. The code of this component is not extensive, so you can understand it in a short time.
Okay, now you have a direction for reading, but you may still encounter a difficulty: what if you can’t understand it?
Although I have provided some methods for reading source code, understanding code written by others is indeed a challenging task. Therefore, my answer is “Don’t worry!”
At the beginning, you may have no idea of what a certain method or class does, and this is a very normal phenomenon. At this time, what you need to do is to have a preliminary understanding of the code structure and outline, and know which classes there are and what APIs they define. This is like learning to write. By making efforts to understand code outside of your knowledge domain, you can expand the scope of your coding skills. If you always stay in the simple, dry, and easily understandable code ocean, it is difficult to truly make progress.
At the same time, don’t imagine that you only need to read the code once! When it comes to reading source code, it takes many repetitions to bring about a “qualitative” change.
I don’t know if you have heard of the “onion code reading method”. It means that when we read source code, we should peel it like an onion. The first reading is only for grasping the overall project structure and outline, and the subsequent multiple readings gradually reveal the essence of the code. Remember: peel only one layer at a time.
In addition to this, if there is another method I keep to myself, it is to personally modify the source code.
In fact, you can truly experience the joy of reading source code only when you make some improvements or changes to the code you are reading. Therefore, when reading the source code of Kafka or any other framework, you may try to modify the code yourself, conduct some experiments, and debug to experience the changes you make have on Kafka. Trust me, you will fall in love with this process.
Finally, I want to emphasize one point: don’t be afraid. Fear is the number one obstacle that prevents many people from reading source code. Just thinking about the thousands of lines of code to read, people instinctively raise the flag of giving up. In fact, no matter how grand a framework is, it is built with lines of code. I always adhere to a seven-word maxim: If you dare write, I dare read! Now I pass it on to you, hoping that you can use it to encourage yourself when you feel like giving up.
Okay, let’s end this season of the course here. There aren’t too many fancy expressions today, just some practical advice. In fact, all I want to say is that we have just started our journey of learning source code.
Reading source code is like a long mountaineering journey. It’s crowded on the halfway point, but we will meet at the summit.
I have prepared a closing questionnaire for you, and I hope you can share your thoughts on this course in the questionnaire. Please click on the image below to fill out the questionnaire in 1-2 minutes. Looking forward to hearing your candid feedback.
](