00 Opening Words Starting Volumetric My SQL Architecture Design From Business Development

00 Opening Words - Starting Volumetric MySQL Architecture Design from Business Development #

Hello, I’m Jiang Chengyao (commonly known as Bankrupt Programmer), currently the Director of Tencent Financial Data Platform and R&D Center.

I have been associated with MySQL for more than ten years. Initially, I started my database career at Joyo.com, then worked at NetEase, where I was responsible for database kernel and cloud database development. Currently, at Tencent, I am in charge of database development for financial payment systems.

Since graduating, I have been involved in various MySQL-related work, including operations, platform development, kernel development, and cloud computing development. I have experienced many sleepless nights as a DBA and accumulated numerous practical architectural experiences.

My Journey with MySQL #

During my time at Joyo.com, I was responsible for the popular online game “Audition.” As a DBA for “Audition,” I was highly esteemed. However, who could have expected that I would encounter an incident where I had to work continuously for 72 hours to recover the game’s data. To prevent such events from happening again, as early as 2008, I designed a multi-instance high availability architecture at Joyo.com. I also utilized LVM snapshot functionality to prevent potential data discrepancies caused by game upgrades.

I can proudly say that I am the earliest DBA in China to work on MySQL kernel. At that time, as the volume of data continued to grow, the business requirements for MySQL databases became more demanding. Not only did we need to use MySQL, but we also had to perform additional kernel development. In response, I delved into the realm of MySQL kernel design and independently developed the SBP (Secondary Buffer Pool) architecture to align with the evolving SSD technology. This architecture was extensively used in Joyo.com, NetEase, and other businesses.

During my time at NetEase, I witnessed the continuous improvement of MySQL’s semi-synchronous replication feature. I foresaw that it would soon enter the core finance business field. As a result, I led the development of the InnoSQL branch version of MySQL at NetEase. I designed and developed the VSR (financial-grade MySQL high availability architecture), which also served as an open-source database component and was successfully applied in the core systems of a major bank.

After joining Tencent in 2017, I led the design and development of the next-generation financial core database architecture. This provided better security for the financial and payment features used by our colleagues.

It can be said that the success of MySQL in internet businesses has greatly benefited me:

The income has been continuously increasing, and compared to other types of databases, MySQL’s income has an obvious advantage. Currently, it’s relatively easy for database professionals in first-tier cities to earn a salary of 500,000 yuan. If you work for an internet company, the salary is practically limitless.
As a career achievement, MySQL has brought me much inspiration. With the rise of the internet, MySQL has become the standard configuration for databases in internet companies. Seeing that the database I operated and developed can support tens of thousands of users is truly an amazing feeling.

I often contemplate how to consolidate the knowledge I have gained over the years in MySQL into a methodology that can be shared, hoping that more students can enjoy the benefits of MySQL’s development.

How to Make the Most of MySQL? #

Although I have published three books in the series of “MySQL Technical Insider” and “MySQL Kernel” over the years, compared to theory, each book’s direction is more specialized and unable to effectively share the implementation of a large-scale internet MySQL architecture from the perspective of the entire business chain.

Similarly, looking at the entire industry, whether it’s technology books or online courses, they all focus on how to use a specific feature of MySQL well, without considering from a business perspective how to design a massive concurrent business MySQL database architecture.

For example, from a theoretical perspective, there is no problem using an auto-incremented INT type for a primary key. However, from the perspective of businesses like Double 11 or Spring Festival red packets, using INT type as the core business primary key is highly undesirable and can even lead to endless troubles.

Another example is that while you may know that MySQL replication technology can build a highly available architecture, is it necessary to rely solely on the database layer to achieve high availability when it comes to different business types? Not necessarily! Because the database has state, when problems occur, the switch speed may be relatively slow. In fact, you can design a high availability architecture by combining it with the business.

In short, when it comes to databases, it is important to closely integrate with the business, think from the perspective of the business’s entire process, and only then can you design a truly useful database architecture.

I am very fortunate to have received an invitation from the Lagou Education platform, which has given me the motivation to create MySQL content again. On Lagou’s website, there are many technical professionals with a desire for knowledge who want to learn courses that are truly helpful to their work.

Therefore, I will focus on creating this course, combining the pain points you encounter when learning MySQL, such as integrating MySQL 8.0’s new features with the business, designing high availability for financial-grade databases, and distributed architecture design. I will leverage my over 15 years of frontline MySQL work experience to help you view database systems from the perspective of the entire business process and design a massive concurrent system based on MySQL. Additionally, I hope that after completing this course, you will develop your own database architecture methodology and actively engage in communication and exploration to continuously grow.

Course Design #

In general, I will work with you gradually and progressively to build a distributed MySQL architecture that can support massive concurrent access through table structure design, index design, high availability architecture design, and distributed architecture design.

Module 1: Table Structure Design. In this module, I will analyze different field types and then teach you about table design in MySQL, such as table structure design, access design, and physical storage design, using actual business cases as examples. By addressing your pain points in table structure design in this module, you will establish the foundational work for architectural design.

Since single table design is not enough to support business deployment, Module 2 is about index design. In this module, I will start with the basic principles of indexes and gradually move on to index creation and optimization. Finally, I will cover complex SQL index design and optimization, such as multi-table joins, subqueries, and partitioned tables. I hope that after completing this part, you will be able to solve all SQL problems in a live environment, whether they are OLTP or complex OLAP businesses.

After covering table structure and index design, a crucial step for business deployment is high availability. MySQL, as an open-source database, provides many high availability solutions, but they all have some issues to varying degrees. Therefore, Module 3: High Availability Architecture Design will guide you step by step to build a complete, reliable, and adaptable high availability solution for various business types.

In addition to high availability, managing massive businesses also requires dealing with the challenges of distributed architecture. For current business and DBA professionals, this is a highly challenging technical issue. In Module 4: Distributed Architecture, I will cover distributed architecture overview, distributed table structure design, distributed index design, distributed transactions, and more. Believe me, after completing this part, you will find that distributed architecture is not as difficult as it seems, and you will be able to address various challenges in distributed architecture without any difficulty.

Module 5: Expansion-oriented is an analysis of popular topics in database design. After learning the content of the first four modules, advancing to study these issues will enable you to design a better architecture from a broader and higher-level perspective and solve corresponding problems, such as hot update issues and data migration.

Overall, this course is worth looking forward to and is what I believe to be the most practical MySQL course in terms of architectural practice. So, I hope you will study this course seriously and thoroughly, and become a qualified database architect as soon as possible.