Special Release One Classic Kafka Learning Materials Available #
Hello, I am Hu Xi. Our course has been updated for some time now. Have you been keeping up with the lessons? If you have been studying along, I believe you have gained a lot.
Of course, I also understand that while learning the source code, there may be some pain amidst the joy of progress. After all, the source code is not easy to grasp.
If you encounter any problems during your studies, feel free to leave me a message. I will reply as soon as possible to help you solve the problem. If you find yourself surrounded by negative emotions, besides persevering, I suggest you try to temporarily escape from these emotions and shift your focus to some lighter topics. Today, the content of this special episode is very relaxing, as I will share with you some Kafka learning materials.
In fact, during this period of updates, many students have asked me, “Teacher, I want to learn more about Kafka. Can you recommend some relevant learning materials to me?” Today, during this special episode, I have specifically gathered various Kafka learning materials, including books, videos, blogs, and all other audiovisual materials. I have compiled them into a list to share with you.
These materials vary in their depth and focus. Some emphasize basic theory, while others delve into the underlying architecture. Some focus on practical examples, while others share best practices.
If you are looking for practical guidance, then you should pay close attention to the community-maintained documentation and various Kafka practical books mentioned by me. If you have a strong interest in studying Kafka source code, then you should focus on the blogs and video materials by various experts mentioned in this lesson. They often provide code-level analysis, and studying these materials can not only broaden our thinking and perspective, but also deepen our understanding of the source code knowledge. It can be said to have multiple benefits.
In short, I recommend that you use these materials in a targeted manner based on your learning goals and interests.
I have divided this list into two main parts: English materials and Chinese materials. I will first provide you with the list of all the English materials I have collected.
English Resources #
I don’t know if you have carefully read the text on the official website. All the content there is written by Kafka Committer, with concise and comprehensive language, and rich and detailed content. I recommend that you pay special attention to the Configuration section, Operations section, and Security section, especially the parameter part in the Configuration section. Mastering these key parameter configurations is a necessary condition for advanced Kafka learning.
- Official documentation maintained by Confluent (link)
Confluent is a commercial company created by the Kafka founding team, mainly providing commercial solutions based on Kafka. We often refer to the products they provide as Confluent Kafka. In my personal opinion, the quality of their official website is better than the community version (i.e., Apache Kafka official website), especially in terms of Security and Kafka Streams introduction, which is obviously superior to the community version. Therefore, I recommend that you focus on learning the documentation on Security configuration and Kafka Streams components on the Confluent official website.
- Kafka Jira list, also known as the bug list
You can search for the Kafka exception names you encounter in the actual environment on this page, and then determine, based on your Kafka version, if the exception is caused by a known bug, thus avoiding wasting more time locating the cause of the problem. In addition, you can contribute to the community by claiming Jira tickets. I will give you a separate class later to introduce the complete process of contributing code to the community.
- Kafka KIP website
KIP stands for Kafka Improvement Proposals, which are new feature proposals for Kafka. Here you can learn about all the proposals and related discussions about new features in the Kafka community. Interestingly, some of the discussions about KIPs are more exciting than the KIPs themselves. For various new features, developers from around the world think actively, discuss with each other, sometimes collaborate and benefit, and sometimes argue fiercely. Another great charm of KIPs is that they are very democratic - anyone can apply for a new feature proposal and put their ideas into practice.
- Kafka section on Stack Overflow
The famous Stack Overflow website needs no further introduction. There are various Kafka questions on it, with varying difficulties, but you can usually find the answers you need. At the same time, if you know Kafka very well, you can try to answer some questions, which not only earns you some points, but also allows you to communicate with Kafka users worldwide, a win-win situation.
The articles here are of high quality, and you must read them carefully. Let me give you a simple example. I have been asked this question by too many people: “How many partitions can a Kafka cluster support?” In fact, we all know that there is no standard answer to this kind of question, but what you need to understand is the principle behind it! Coincidentally, there is an article on the Confluent blog that explains this principle in detail, written by Jun Rao, one of the founders of Kafka. You might as well take a look at it.
- Various non-public Kafka materials in the community, including the community-maintained Confluence documentation and Google Docs
You can find almost all Kafka design documents in the Confluence Space, among which the articles on the Controller and the design of the new Clients are particularly worth reading. Google Docs mainly include two articles: one is the detailed design document for Kafka transaction, and the other is the design document for Controller rewrite. These two articles are the most detailed Kafka design documents I have seen so far. Many Kafka books and blogs in China are mostly cited or directly translated from these two articles, which shows their extraordinary value.
- Kafka community’s Twitter page and Confluent’s Twitter page
You might say, what kind of learning material is Twitter? But in fact, many times, you can find valuable Kafka articles on it, especially Confluent’s Twitter, which regularly tweets about Kafka best practices. Every time I see these articles, I feel like I’ve discovered a treasure. Let me give you an example. On October 12, 2019, Kafka Twitter retweeted an article named Exploit Apache Kafka’s Message Format to Save Storage and Bandwidth. The content in it is of high quality and very enjoyable to read. I suggest you read it carefully.
- Kafka videos on YouTube
These videos mainly include explanations of Kafka principles, and sharing by industry experts, etc. Sometimes, you will find that our understanding of textual materials is not as in-depth as watching videos. If your English is good, I recommend that you pay special attention to these sharing videos on YouTube.
Alright, the above nine resources are the English learning materials that I have summarized for Kafka. Overall, these resources do not require a high level of English proficiency. Even for English videos on YouTube, real-time translation is supported, so you don’t have to worry about not understanding the content.
Next, I will provide a list of Chinese resources.
Chinese Resources #
First, I will recommend five books on Apache Kafka that I think are good.
This book is the Chinese translation of “Kafka: The Definitive Guide”. I have read both the Chinese and English versions, and I must say that the translation of the Chinese version is excellent. This book, although thin, covers almost all aspects of Kafka, and it is written by Committers, so the quality is impeccable.
After its publication, this book quickly became the best book on Kafka available in the market. It truly lives up to its name “Technical Insider”. The explanations of many Kafka principles in this book are clear and in-depth, and I personally found it very informative.
I know the author of this book personally. He is a master of both Kafka and RabbitMQ, and is well-known in the field of message middleware. This book was published in 2019 and is currently the latest book on Kafka in the market. I recommend you to read it.
This book is the Chinese translation of “Kafka Streams in Action” and is written by a Kafka Committer. It is the most in-depth book on Kafka Streams that I have seen. If you want to learn real-time stream processing based on Kafka Streams, then this book is a must-read.
This book, which I wrote, was published in 2018. Unlike the previous Chinese books that focus on Kafka design principles, this book mainly discusses practical applications of Kafka. In this book, I share various “tricks of the trade” that I have accumulated over the years of participating in the Kafka community and using Kafka. I recommend you to take a look at this book if you are more interested in practical usage.
That concludes the book recommendations. Now let me introduce three websites to you.
The first one is OrcHome. As far as I know, OrcHome is a Kafka tutorial website maintained by Chinese developers. The most distinctive feature of this website is its Kafka Q&A section, where you can ask questions and there are dedicated people who will answer your questions.
The second one is my own blog. Most of the articles on this blog are original content about Kafka or other big data technologies. Additionally, I regularly share and repost excellent blogs from both domestic and international sources.
The third one is the Kafka section on Zhihu, a Chinese knowledge-sharing platform. Unlike Stack Overflow, the questions in this section are mainly focused on theoretical discussions. Usually, people are quite active in answering these kinds of questions, and I frequently respond to questions here as well. If you want to delve deeper into certain Kafka principles, you can consider asking your questions in this Kafka section on Zhihu. I am confident that you will receive prompt responses.
Summary #
Well, the content above is the list of Kafka learning materials that I have summarized. I have organized them into a table for you to focus on.
Additionally, I strongly recommend that you carefully read through the community’s official website and the Confluent documentation. I guarantee that you will gain a lot from them. After all, compared to the other items on the list, the official website documentation is the most authoritative and original source of information.
Post-Class Discussion #
Lastly, I would like to ask you to share your Kafka learning booklist, websites, audio-visual materials, and effective learning methods.
Feel free to express your thoughts and discuss with me in the comments section. You are also welcome to share today’s content with your friends.