Closing Words - Take Dreams as Horses, Don’t Worry About the Distance #

Hello, I am Hu Xi. Today, my column “Core Techniques and Practices of Kafka” officially comes to an end.

Looking back at the past few months we spent together in this column, my heart is filled with a sense of accomplishment. It goes without saying that I typed every word of these 42 lectures and spent more than ten hours recording them. Not to mention the active and enthusiastic questions and discussions in the comment section, all of which have deeply touched and benefited me greatly.

At this moment, a thousand words converge into one phrase: “Thank you!” Thank you for your support to me and this column, thank you for your encouragement and questions, and thank you for your recognition and expectations. Additionally, I want to congratulate you for completing all the courses in this column. Your persistence and dedication are admirable.

Although the updates of this column have come to an end, I believe our journey of learning Kafka will not. On the contrary, for you, it may be a new beginning.

Do you remember the opening words: “Stay focused and work hard!” I have always believed that learning any technology, or even mastering anything, as long as you put in the effort, theoretically you can disregard all learning methods or shortcuts. However, if you ignore perseverance and persistence, no amount of crash courses can lead you to the heights you desire. The famous “10,000-hour rule” explicitly states that “10,000 hours of practice is the necessary condition for anyone to become a world-class master.”

Again, as long as you persist in investing time into learning, you can become an expert in a certain field. Therefore, from a certain perspective, the ingredients of this “chicken soup” are very simple, just four words: “Just do it.”

Does this mean that books or columns summarizing other people’s wisdom are useless? Of course not! The gems of others can be used to polish our own skills. The greatest role of books and columns is that when you encounter a crossroad, they can help you quickly identify known obstacles on the road ahead, allowing you to take fewer detours and achieve your goals faster. But the premise is that you have to be on the road, not just relying on them for a quick fix.

At the end of this column, I would like to share some personal experiences in learning big data frameworks with you. These experiences are not only applicable to learning Kafka, but also to learning other frameworks and even distributed systems.

First and foremost, the most important thing is to have a solid foundation of technical skills. This is the cornerstone for us IT practitioners to survive.

This foundation encompasses many aspects, such as operating systems, data structures, etc. However, what I want to emphasize is mastery of the Java language.

Currently, many big data frameworks are developed using Java or JVM-based languages. Therefore, being proficient or even mastering Java is the cornerstone for learning big data frameworks! True mastery not only requires you to be proficient in Java coding, but also demands a detailed understanding of the underlying JVM. In terms of learning at this level, I would like to give you three recommendations.

1. Continuously improve your Java foundation. For example, you can read the Java Language Specification and JVM Specification on the official Java website. Many people do not attach much importance to language specification documents, but in fact, there are relevant explanations in the Java Language Specification regarding knowledge about threads and synchronization in Java.

2. Enhance your Java multi-threading and I/O development capabilities. Many underlying implementations of big data frameworks extensively utilize the multi-threading capabilities and NIO of Java to achieve their functionalities. Take Kafka as an example, it not only requires multi-threading, but also heavily relies on NIO for network communication. Therefore, you must be proficient in these areas.

3. Master JVM tuning and garbage collection. I recommend reading the book “Java Performance”. Although most GC collectors have evolved into the G1 era, many of the tuning techniques in the book are still applicable. When tuning the JVM for Kafka, we also rely on this knowledge for guidance. In addition, you need to learn the design of distributed systems.

Many classic problems in the field of distributed systems are the problems that need to be faced and solved when designing and developing any distributed system, such as the well-known consistency problem, leader election problem, partitioning and replication problem, etc. These problems are also reflected in Kafka, and we have covered them in this column. Therefore, the foundational concepts of distributed systems are important factors in helping you master big data distributed frameworks in the future.

Moreover, many classic distributed problems have been researched for many years in the industry, and there are abundant records of both theory and practical examples. For example, the concept of partitioning, which we talked about earlier in this column, is not a new concept in the design of distributed systems. As early as the 1960s and 1970s, industry experts were studying the implementation of partitioned databases. To better understand partitioning or sharding in big data frameworks, it is impossible to ignore the partitioning and partitioning mechanisms in distributed systems.

These experiences are more focused on theoretical learning. Don’t underestimate the value of theory, after all, Lenin said, “Without revolutionary theory, there can be no revolutionary movement.” The “movement” here refers to practice. Let theory guide practice first, and then complement theory with practice. This is the best principle for learning anything successfully.

After emphasizing theory, it naturally leads to practice. The practice I mentioned here is not just about simple usage of frameworks. Downloading Kafka from the official website, starting it, creating a producer and a consumer, and successfully sending messages end-to-end, this is not called practice, it is simply application. True practice must include your own thinking and verification, and it should be tied to real business scenarios. I’m not against simply learning a framework, but based on my personal experience, learning in practical work often leads to the fastest and most solid understanding.

In addition, during the actual learning process, it is best to record every detail of the problems you encounter and how you solve them, and constantly accumulate. You should know that many experts in technology become experts not only because they have a solid understanding of theory and have encountered many “pitfalls”, but more importantly, because they don’t repeat the same mistakes.

Confucius once praised his disciple Yan Hui for “not making the same mistake twice.” “Not making the same mistake twice” means not repeating mistakes. In my opinion, in terms of practice, someone who doesn’t repeat the same mistakes can already be called an expert.

Well, these are the experiences I have summarized in the process of learning technology. Now I share them all with you, hoping to be helpful to you.

I have prepared a closing questionnaire for you. The questions are not many, and it will only take two to three minutes to complete. I hope you can speak freely and express your true learning experiences and opinions. I will definitely read it carefully and look forward to your feedback.

](https://jinshuju.net/f/FFQ5PT)

This is the final issue, and I also invite you to summarize your learning in this column here. If you have any questions about the column or any other content, feel free to ask, and I will continue to reply to your messages.

Farewell, my friend, for a thousand miles. As the saying goes, “The first ninety miles are the hardest,” I hope you continue to delve deep into the path of technology and achieve what you desire, not wasting the beautiful time. Until we meet again!