13 The Use of Cache Scenario I - How to Choose the Caching Read-Write Strategy #
In the previous lesson, I introduced the definition, classification, and limitations of caching. You should now have a preliminary understanding of caching. Starting today, I will guide you through the correct way of using caching, such as the read and write strategies of caching, how to achieve high availability of caching, and how to deal with cache penetration. By understanding these contents, you will have a deep understanding of caching usage, so that you can handle caching easily in your actual work.
Today, let’s talk about the read and write strategies of caching. You may think that caching is simple - just read from cache first and if cache miss, query from the database and then store the result in cache. In reality, different business scenarios require different read and write strategies for caching.
When selecting the strategy, we also need to consider various factors, such as the possibility of dirty data being written to the cache, the read and write performance of the strategy, and the possibility of decreasing cache hit rate, etc. Next, I will use the standard “cache + database” scenario as an example to analyze classical cache read and write strategies and the scenarios they are suitable for. This way, you can choose different read and write strategies according to different scenarios in your daily work.
Cache Aside Strategy #
Let’s consider a simple business scenario where you have a user table in your e-commerce system. The table only has two columns: ID and age. In the cache, we store the age information of users with their IDs as keys. Now, suppose we want to change the age of user with ID 1 from 19 to 20. How should we do it?
You might come up with the following approach: First, update the record in the database with ID 1, and then update the data in the cache with key 1.
This approach can cause inconsistency between the cache and the database. For example, while request A is updating the age of the user with ID 1 in the database from 19 to 20, request B also starts updating the data of user with ID 1. It changes the age in the database to 21 and then updates the age in the cache to 21. Then, request A proceeds to update the data in the cache and changes the age to 20. At this point, the age in the database is 21, but the age in the cache is 20.
Why does this problem occur? It’s because updating the database and updating the cache are two independent operations, and we haven’t implemented any concurrency control mechanisms. Therefore, when two threads concurrently update them, data inconsistency can occur due to the different order of writes.
Furthermore, directly updating the cache also leads to another problem - update loss. Let’s take the account table in our e-commerce system as an example. Assume the cache stores complete account information, including fields for ID, username, and balance. When updating the balance in the cache, you need to retrieve the complete account data from the cache, update the balance, and then write it back to the cache.
This process can also have concurrency issues. For instance, if the original balance is 20 and request A reads the data from the cache, adds 1 to the balance, and changes it to 21, before writing it back to the cache, there’s another request B that also reads the data from the cache, adds 1 to the balance, and changes it to 21. Both requests simultaneously write the balance back to the cache. As a result, the balance in the cache becomes 21, but we actually expect it to be increased by 2, which is a significant problem.
So how can we solve this problem? By not updating the cache directly when updating data but by deleting the data in the cache instead. Then, when reading data, if the data is not found in the cache, retrieve it from the database and update the cache.
This strategy is the most common approach when using caches, known as the Cache Aside strategy (also known as the Bypass Cache strategy). With this strategy, the data in the cache is based on the data in the database, and the data in the cache is loaded on-demand. It can be divided into read and write strategies. The steps for the read strategy are as follows:
-
Read data from the cache.
-
If the cache is hit, return the data directly.
-
If the cache is miss, query the data from the database.
-
After retrieving the data, write it to the cache and return it to the user. The steps for writing a strategy are as follows:
-
Update records in the database;
-
Delete cache records.
You may wonder, can we delete the cache first and then update the database in writing a strategy? The answer is no because this may also result in inconsistent cache data. Let me explain this using the scenario of a user table.
Suppose a user’s age is 20 and request A wants to update the user’s age to 21, so it will delete the content in the cache. At this time, another request B wants to read the user’s age. It queries the cache and finds no match, so it reads the age as 20 from the database and writes it back to the cache. Then request A continues to update the database, changing the user’s age to 21, which causes inconsistency between the cache and the database.
So, is it perfectly fine to use strategies like Cache Aside, where we first update the database and then delete the cache? In theory, it still has flaws. Suppose a user’s data does not exist in the cache, and request A reads the data from the database and finds the age as 20. Meanwhile, before it writes to the cache, request B updates the data. It updates the age in the database to 21 and clears the cache. Now, request A writes the data it read from the database (age = 20) to the cache, causing inconsistency between the cache and the database.
However, the probability of such problems occurring is not high. This is because cache writes are usually much faster than database writes. So, it is unlikely in practice that request B has already updated the database and cleared the cache while request A is still updating the cache. And once request A has updated the cache earlier than request B clears the cache, the subsequent requests will reload the data from the database because the cache is empty. Therefore, this inconsistency does not occur.
Cache Aside is the most commonly used cache strategy in our daily development work, but we also have to adapt it based on different situations. For example, when registering a new user, following this update strategy, you would write to the database and then clear the cache (if there is any data to clear in the cache). However, if I immediately read the user information after registering a user and if the database is master-slave replicated, there may be cases where the user information is not read due to the delay between the master and the slave database.
The solution to this problem is to write to the cache after inserting new data into the database. This way, subsequent read requests will retrieve the data from the cache. And since it concerns a newly registered user, there will not be any concurrent updates to the user information.
The biggest problem with Cache Aside is that when there are frequent writes, data in the cache will be frequently cleared, which can have some impact on the cache hit rate. If your business has strict requirements on cache hit rate, you can consider two solutions:
-
One approach is to update the cache while updating the data, but first add a distributed lock before updating the cache. This way, only one thread is allowed to update the cache at the same time, avoiding concurrency issues. However, this may affect the performance of writing.
-
Another approach is also to update the cache while updating the data, but set a relatively short expiration time for the cache. This way, even if there is inconsistency in the cache, the data in the cache will expire quickly, and the impact on the business will be acceptable.
Of course, besides this strategy, there are several other classic cache strategies in the field of computer science, each with their own applicable usage scenarios.
Read/Write Through Strategy #
The core principle of this strategy is that users only interact with the cache, and the cache communicates with the database to write or read data. It’s like reporting your work directly to your immediate superior, who then reports it to their superior. You cannot bypass levels in reporting.
The Write Through strategy works as follows: First, check if the data to be written exists in the cache. If it does, update the data in the cache and synchronize it with the database through the cache component. If the data is not in the cache, we call this a “Write Miss”.
Generally, we have two options for handling a “Write Miss”: one is “Write Allocate”, which means writing the data into the cache and synchronously updating it to the database through the cache component; the other is “No-write allocate”, which means bypassing the cache and directly updating the database.
In the Write Through strategy, we usually choose “No-write allocate” because regardless of the chosen “Write Miss” method, we need to synchronize the data update with the database. “No-write allocate” reduces one cache write compared to “Write Allocate” and improves write performance.
The Read Through strategy is simpler. The steps are as follows: First, check if the data exists in the cache. If it does, return it directly. If it doesn’t, the cache component is responsible for synchronously loading the data from the database.
Here is an illustration of the Read Through/Write Through strategy:
The characteristic of the Read Through/Write Through strategy is that the cache nodes, not the users, communicate with the database. This strategy is less commonly seen in our development process compared to the Cache Aside strategy because the distributed cache components we often use, such as Memcached and Redis, do not provide features that write to the database or automatically load data from the database. However, this strategy can be considered when using local caches, such as in Guava Cache’s Loading Cache mentioned in the previous section.
In the Write Through strategy, writing to the database is synchronous, which can have a significant impact on performance. Writing to a database synchronously has a higher latency compared to writing to a cache. Can we update the database asynchronously? This leads us to the “Write Back” strategy which we’ll discuss next.
Write Back Strategy #
The core idea of this strategy is to only write data into the cache when writing, and mark the cache block as “dirty”. The dirty block will only write its data to the backend storage when it is used again.
It should be noted that in the case of “Write Miss”, we adopt the “Write Allocate” approach, which means that we need to write the data into the cache as well as the backend storage. This way, we only need to update the cache in subsequent write requests, without updating the backend storage. I’ve included a diagram of the Write Back strategy below:
If Write Back strategy is used, there are some changes to the read strategy as well. When reading from the cache, if the cache hit is found, the cache data is returned directly. If the cache is not hit, a available cache block is searched. If the cache block is “dirty”, the previous data in the cache block is written to the backend storage, and the data is loaded from the backend storage into the cache block. If the cache block is not dirty, the cache component loads the data from the backend storage into the cache. Finally, the cache is set as not dirty and the data is returned.
Have you noticed? Actually, this strategy cannot be applied to common database and cache scenarios. It is a design in computer architecture, such as when writing data to a disk. Whether it’s the operating system’s Page Cache, asynchronous flushing of logs, or asynchronous writing of messages in message queues, most of them adopt this strategy. The performance advantage of this strategy is undeniable, as it avoids the random write problem caused by directly writing to the disk. After all, the delay of writing to memory and writing to disk differs by several orders of magnitude.
However, because caches usually use memory, and memory is not persistent, if the cache machine loses power, the dirty data in the cache will be lost. So you may find that after a system power failure, some of the files previously written will be lost, which is caused by Page Cache not having time to flush to the disk.
Of course, you can still use this strategy in some scenarios. When using it, I would like to offer you some implementation advice: when writing data to a low-speed device, you can temporarily store the data in memory for a period of time, even do some statistical summaries, and then periodically flush it to the low-speed device. For example, when measuring the response time of your interface and need to write the response time of each request to the log, you can collect the logs by the monitoring system and do the statistics later. But if you log every request, it will undoubtedly increase disk I/O. In this case, it’s better to temporarily store the response time for a period of time, perform simple statistical averaging, count the number of requests within each time interval, and then periodically batch log them.
Summary of the Lesson #
In this lesson, I mainly introduced several caching strategies and the corresponding use cases for each strategy. The key points I want you to grasp are:
-
Cache Aside is the most commonly used strategy when using distributed caching, and you can directly use it in your actual work.
-
Read/Write Through and Write Back strategies require support from the caching component, so they are more suitable for use when implementing a local caching component.
-
Write Back strategy is a strategy in computer architecture, but the strategy of only writing to the cache and asynchronously writing to the backend storage has many application scenarios.
Furthermore, you need to understand that the strategies mentioned today are standard practices and need to be flexibly used or even modified based on the actual business characteristics in the actual development process. These business characteristics include but are not limited to: the overall data volume, the read-to-write ratio of the access, tolerance for data inconsistency time, and requirements for cache hit rate. By combining theory with practice and analyzing each specific situation, you can come up with a better solution.