01 How to Deal with Low Performance in Business Data Access #

Hello, I am your caching teacher, Chen Bo. Welcome to the first lesson “Principles of Caching”. This lesson mainly covers the basic ideas of caching, the advantages of caching, and the cost of caching.

Definition of Caching #

Let’s first look at the definition of caching.

• The original meaning of caching is the RAM (Random Access Memory) used to accelerate CPU data exchange. Usually, this kind of memory uses more expensive but faster Static RAM (SRAM) technology to accelerate DRAM. This is a narrow definition of caching.
• The broad definition of caching is more general, any storage medium that can be used for high-speed data exchange is considered caching, whether it is hardware or software.

The purpose of caching is to solve the problem of high cost in obtaining original data by creating a new data exchange buffer, making data access faster. This lesson mainly focuses on the broad definition of caching, especially various caching components and technologies widely used in Internet products.

Caching Principle #

Basic Idea of Caching #

The basic idea of caching is to use the principles of temporal locality, trading space for time to accelerate data retrieval. However, due to the high cost of caching space, the trade-off between access latency and cost must be considered in practical architectural designs. There are three key points here.

• The first is the principle of temporal locality, which means that data that has been accessed once will be referenced multiple times in the future. For example, after a Weibo post is read by one person who found it interesting, it is likely to be read by more people. Of course, if it becomes a popular Weibo post, it will be viewed by millions or tens of millions of users.
• The second is trading space for time. Because the original data retrieval is too slow, we allocate a high-speed independent space for efficient access, thus accelerating data retrieval.
• The third is the trade-off between performance and cost. When building a system, we hope that the system has higher access performance and lower latency. However, as the performance and latency become lower with the same data storage and access scale, the cost also increases. Therefore, when designing system architecture, you need to make trade-offs between system performance and development and operation costs. For example, in the figure on the left, SSD (Solid-State Drive) has a capacity 10 to 30 times larger than memory for the same cost, but the read-write latency is 50 to 100 times higher.

Advantages of Caching #

The main advantages of caching are as follows:

• Improved access performance
• Reduced network congestion
• Reduced service load
• Enhanced scalability

As introduced earlier, caching stores original data, which can significantly improve access performance. However, in actual business scenarios, the data stored in the cache is often intermediate data that needs to be accessed frequently or even the final result. These data are much smaller than the original data in the database, which reduces network traffic and congestion. Additionally, due to the reduced need for parsing and computation, the load on the service caller and storage service can also be significantly reduced. Caching has high read-write performance and fast warm-up time. In cases of performance bottlenecks in data access or sudden surge in traffic, it can be quickly deployed and put into operation. Similarly, once the traffic stabilizes, it can be taken down at any time, thus greatly enhancing the scalability of the system.

Cost of Caching #

However, unfortunately, everything has two sides, and caching is no exception. While enjoying the benefits brought by caching, we are also destined to pay a certain cost.

• First of all, introducing caching into the service system will increase its complexity.
• Secondly, because caching is more expensive than storing data in the original database, the cost of system deployment and operation is also higher.
• Finally, as the same piece of data exists in both the cache and the database, and there may even be multiple copies of the data within the cache, data consistency issues arise. Additionally, the cache system itself may also have availability and partitioning issues. This requires us to strengthen our understanding of caching principles, caching components, and best practices for caching systems. It is important to design caching properly from the beginning of system architecture in order to reduce the side effects of introducing caching and make the caching system a strong foundation for efficient and stable operation of the service system.

Generally speaking, the full-scale original data of the service system is stored in the database (such as MySQL, HBase, etc.), and all data can be accessed through database operations. However, the read and write performance of databases is low, and latency is high. For example, the read and write QPS (Queries Per Second) of a single instance of MySQL is usually only in the range of thousands (3000 to 6000), and the average time for read and write operations is in the range of 10 to 100 milliseconds. If a user request needs to aggregate 20 different pieces of data, the database queries alone would take hundreds of milliseconds or even seconds. On the other hand, the read and write performance of caches can make up for the shortcomings of databases. For example, the read and write QPS of Memcached can reach millions, and the average time for read and write operations is below 1 millisecond. Combined with concurrent access technology, even if a single request needs to query hundreds of pieces of data, it can easily handle it.

However, cache capacity is limited, and it can only store a portion of frequently accessed hot data. At the same time, the same piece of data may exist simultaneously in both the cache and the database. If not handled properly, data inconsistency issues may occur. Therefore, when processing business requests, the service system needs to design the way cache is read and written appropriately. The goal is to ensure efficient data delivery while avoiding various issues such as data inconsistency.

Alright, that wraps up all the content for the first lesson. Let’s do a quick review together. First, in this lesson, you learned about the definition and basic ideas of caching. Then, you learned about the advantages and costs of caching.