User Story - Endless Possibilities from the Heart #
Hello, I am Longslee, and I am thrilled to learn “40 Questions on High-Concurrency System Design” with all of you.
I have been working in the software industry for nine years. I used to work for a company in the taxation industry and currently work for a telecommunications-related company, focusing on development and operations.
I can’t say I am an old user of “Geek Time” because I have only been exposed to it for a few months. Initially, I approached “Geek Time” with a trial mindset and subscribed to a few courses. Later on, it naturally became a necessary resource for me to obtain information during my spare time.
I came across this course in October this year. At that time, when I casually opened “Geek Time,” I noticed the course “40 Questions on High-Concurrency System Design” and was attracted by the title of the opening article, “Why do you need to learn about high-concurrency system design?” The opening article mentioned:
“Just because the company’s business traffic is stable does not mean that it will not encounter high-concurrency requirements. In order to avoid being caught off guard when encountering problems, it is necessary for you to accumulate enough knowledge on high-concurrency in advance, so as to have the ability to respond to high-concurrency requirements at any time.”
These words hit home for me.
Recalling my own industry, it is a very traditional IT industry that is hardly related to the “Internet.” Therefore, it is particularly difficult for me to have direct contact with cutting-edge technology stacks. However, even though the industry is traditional, it does not prevent the occurrence of high-concurrency in daily work. For example, occasionally, there are online promotional activities.
From my personal perspective, besides being attracted by the opening article, I chose this course because I wanted to broaden my horizons and tap into my potential. On the other hand, when faced with high-concurrency situations, I didn’t want to be at a loss or have a blank mind.
In terms of course design, each lesson’s title ends with a question mark. This seemingly ordinary design easily connects my actual work with the learning process. I would ask myself some questions first, such as “Why do we need to architect in layers?” or “How do we avoid message duplication?” After I came up with some answers, I would then continue with the formal learning process, filling in the gaps in conceptual knowledge.
Personally, I think this is also a small feature of this course. Instructor Tang Yang raises questions and answers them based on his experience, creating a good closure for each article.
Currently, in my industry and project, in order to deal with increasingly complex business scenarios and frequent promotional activities, transformations are gradually taking place. More internet industry knowledge is being introduced, and products are becoming more up-to-date.
As a member of this industry, I naturally encounter difficulties and bottlenecks in my daily work. However, when looking for solutions, I often limit myself to the technology system that I am good at and past experiences. After learning this course, I have broadened my horizons and involuntarily started to think, “Can I solve certain problems using what I learned today?” or “Were the middleware and usage patterns I chose reasonable?” and so on.
Also, as I mentioned, the industry I am in is constantly changing. In fact, from the current trend, it is becoming increasingly difficult to define between traditional information products and mainstream internet products. For example, although the school’s management system may seem different from various websites we encounter, how many course selection systems can handle the instant surge in traffic when classes start?
“The 17th lesson, ‘Message Queues: How to Handle Tens of Thousands of Order Placement Requests per Second during Flash Sales?’” talks about the “tactics” used by various companies to handle foreseeable and short-term high traffic. In my opinion, these “tactics” can also be applied to the course selection system in universities. Because the school’s management system is usually idle, upgrading the overall QPS is not cost-effective. Therefore, as long as the stability of the service during course selection is guaranteed, a message queue can be introduced to alleviate the pressure on the database, and through asynchronous splitting, the processing speed of core business can be improved.
Actually, there are many lessons that left a deep impression on me, such as the 2nd, 10th, and 13th lessons.
Looking only at the title “Lesson 02 | Architectural Layers: Why Do We Have to Do This?”, I would initially think it’s a “hackneyed topic.” Software layering has been used in practical projects extensively. So, why did the instructor dedicate a whole lesson to it? However, when I saw the sentence “If the business logic is simple, can we directly go from the presentation layer to the data access layer, or even directly read the database?” and related it to my own business, it struck a chord.
In one of the projects I participated in, because the business logic was basically the same as the database logic, we interacted directly with the data access layer from the presentation layer. But if the database or data access layer underwent changes, we would have to modify multiple places in the presentation layer. If we accidentally missed the places that needed adjustment, it would be difficult to even identify the problem. Moreover, if a logical layer is introduced later on, modifying the layers would become more complex.
For me, this article touched me and prompted my thinking. Therefore, in future projects, I will persist in using a layered architecture.
Lesson 10, “ID Generator: How to Ensure the Global Uniqueness of IDs After Sharding?”, provided me with ideas for my project: My requirement is not to ensure the uniqueness of primary keys after sharding, but because I need to allocate unique IDs to each client, using client-based strategies would unavoidably result in duplicates. So, when I read:
One approach is to embed the code into the business code, which is distributed across the business servers. The advantage of this approach is that the business code does not need to make cross-network calls when in use, which improves performance. However, it requires more machine ID bits to support more business servers. Additionally, because there are many business servers, it is difficult to ensure the uniqueness of the machine ID, so distributed consistency components like ZooKeeper need to be introduced to ensure that a unique machine ID is obtained each time the machine restarts…
I have adopted a concept similar to an issuing machine and abandoned the UUID-like algorithm used before. By using the distributed ID issued by the issuing machine, the sorting performance of the database has improved, and the business meaning is stronger.
Apart from that, before studying “Lesson 13 | Cache Usage Strategies (Part 1): How to Choose Cache Read and Write Strategies?”, I didn’t consider the consistency between the database and the cache too much in my project. For example, when writing data, I chose to write to the database first and then to the cache, considering that if writing to the database fails and the transaction is rolled back, the cache will not be written. If the cache fails to be written, a retry mechanism can be designed.
It seemed fine, but because I didn’t consider the inconsistency between the two sides in the case of multi-threaded updates, the result was that sometimes the results obtained from the front end did not match the real data. Later, according to the Cache Aside strategy mentioned by Tang Yang, I suddenly realized and applied this strategy to the project, and the effect was good. In this lesson, I have learned a lot from Tang Yang’s personal experience and directly applied it to my own project.
I am really grateful to Tang Yang and I am also very happy to have encountered this course. I would like to express my sincere gratitude here.
So how did I learn this course? Here, I would like to share a few points:
Theory and practice must go hand in hand
After completing the course, in addition to actively thinking about questions like “Can I use it?”, “How to use it?”, and “When to use it?”, one must strike while the iron is hot and either continue to delve into the topic, browse other materials to consolidate it, or implement it in code to turn it into one’s own skill. If time permits, one can even start to improve the project right away.
Comments section = treasure trove area
At the end of each lesson, I would leave questions or share experiences in the comments section. I actually like to ask questions because it is related to an experience I had when I attended a presentation during college. At that time, the person in charge of recruitment was a Taiwanese engineer who had returned from studying in the United States. After introducing the company, he asked if anyone had any questions, but no one answered.
Later, he shared an experience that made me very impressed. He said that when he first went to university in the United States, the professor would answer questions after class, and a white American student asked a question that seemed very simple and childish to Chinese students. After that, in every class, this white student would ask questions, and gradually, he couldn’t understand the questions he asked anymore! Eventually, even the professor didn’t understand.
Therefore, I will continue to ask questions and ask if I don’t understand. I treat the comments section as a treasure trove and look at everyone’s comments to reflect. For example, @李冲 provided several follow-up comments, which clarified my misunderstanding of the Bloom filter and also introduced me to another variant of the Bloom filter, the cuckoo filter.
Taking detailed notes diligently
Sometimes, I understood something thoroughly at the time, but after two or three days, it became a bit vague. So later, I wrote my understanding in the form of a mind map, which could be referred to anytime, anywhere. In addition, corresponding comments can also be written behind the code implementing these solutions, which can help review and refresh memory.
Finally, I would also like to share why I chose this column as a learning method. Those who make good use of search engines must have experienced that the knowledge found is scattered everywhere, with many redundancies, and there are few introductions from experienced ones. I cannot systemize the knowledge I find within a limited time.
Of course, to learn systematically, one can rely on books. But for me, the learning cycle of books is long, and there is little correlation between chapters. It’s easy to forget what was learned later. Books often focus on one professional point, often skipping cross-discipline knowledge. On the other hand, columns have the author’s own understanding, with continuity between them, making learning easier and more enjoyable.
Taking the consistency hash as an example, I read many articles about consistency hashing online, but I didn’t see any application, let alone the flaws in the application. Some descriptions even made me mistakenly believe that when the nodes change, the data will also migrate. Tang Yang’s “Lesson 14 | Cache Usage Strategies (Part 2): How Does Cache Achieve High Availability?” filled in the blind spots I couldn’t see on the Internet. After communicating with the teacher in the comments section, I gained a refreshing sense of understanding.
Of course, these are just my personal feelings, and everyone has their own learning methods. Maybe everyone’s starting points are different. Here, I just want to share my true feelings and sincerely thank everyone for listening to my story.
In summary, there is no shortcut to improving oneself. The only way is to move forward step by step and climb out of the pitfalls encountered.
For me, Tang Yang’s “40 Questions on Designing High-Concurrency Systems” is like a timely rain, filling in the gaps in my knowledge of high-concurrency. I believe that after seriously studying the course, my perspective will definitely broaden, and my professional career will enter a new chapter.