31 Transaction Mechanism Can Redis Achieve ACID Properties #
Transactions are an important feature of databases. A transaction refers to a series of operations that read and write data. During the execution of a transaction, specific properties are provided to ensure its integrity, including Atomicity, Consistency, Isolation, and Durability, also known as the ACID properties. These properties not only require certain results from the transaction execution but also demand specific changes in the database’s data state before and after the transaction.
So, can Redis fully guarantee the ACID properties? After all, if any of these properties cannot be guaranteed in certain scenarios, it may lead to data errors. Therefore, it is essential for us to understand Redis’s support for these properties and prepare strategies accordingly.
Next, let’s first understand the specific requirements that the ACID properties impose on transaction execution. With this knowledge, we can accurately judge whether Redis’s transaction mechanism can ensure the ACID properties.
Requirements of ACID properties for transactions #
Let’s first look at atomicity. The requirement for atomicity is clear, which means that multiple operations in a transaction must either all be completed or all not be completed. Atomicity is also the most valued property when using transactions in business applications.
Let me give you an example. Suppose a user purchases two products, A and B, in an order. In this case, the database needs to deduct the inventory of both of these products. If only the inventory of one product is deducted, then after the order is completed, the inventory of the other product will definitely be incorrect.
The second property is consistency. This is easy to understand, it means that the data in the database is consistent before and after the execution of the transaction.
The third property is isolation. It requires that when a database executes a transaction, other operations cannot access the data being accessed by the transaction.
Let me explain this with the example of a user placing an order. Suppose the existing inventory for products A and B is 5 and 10 respectively, and user X places an order for 3 of A and 6 of B. If the transaction does not have isolation, while the transaction of user X is being executed, user Y also suddenly purchases 5 units of B. This, when combined with the 6 units of B purchased by X, exceeds the total inventory value of B, which does not meet the business requirements.
The final property is durability. After the database executes a transaction, the modifications to the data need to be persistently saved. When the database restarts, the values of the data should be the modified values.
After understanding the specific requirements of ACID properties, let’s take a look at how Redis implements the transaction mechanism.
How does Redis implement transactions? #
The execution process of a transaction includes three steps. Redis provides two commands, MULTI and EXEC, to complete these three steps. Let’s analyze them below.
Step 1: The client needs to explicitly indicate the start of a transaction with a command. In Redis, this command is MULTI.
Step 2: The client sends the specific operations to be executed in the transaction (such as data manipulation) to the server. These operations are the data read and write commands provided by Redis, such as GET and SET. However, although these commands are sent to the server by the client, the Redis instance only temporarily stores these commands in a command queue and does not execute them immediately.
Step 3: The client sends a command to the server to commit the transaction, allowing the database to actually execute the specific operations sent in Step 2. The EXEC command provided by Redis is used to execute the transaction commit. When the server receives the EXEC command, it will actually execute all the commands in the command queue.
The code below demonstrates the process of executing a transaction using MULTI and EXEC, which you can review.
# Start the transaction
127.0.0.1:6379> MULTI
OK
# Decrease a:stock by 1
127.0.0.1:6379> DECR a:stock
QUEUED
# Decrease b:stock by 1
127.0.0.1:6379> DECR b:stock
QUEUED
# Actually execute the transaction
127.0.0.1:6379> EXEC
1) (integer) 4
2) (integer) 9
Let’s assume that the initial values of the keys a:stock and b:stock are 5 and 10, respectively. The two DECR commands executed after the MULTI command are intended to decrement the values of keys a:stock and b:stock by 1. The return results of these commands, QUEUED, indicate that these operations have been stored in the command queue and have not been executed yet. After executing the EXEC command, you can see that it returns 4 and 9, which indicates that the two DECR commands have been successfully executed.
Okay, by using the MULTI and EXEC commands, we can achieve the concurrent execution of multiple operations. However, does this meet the ACID properties required by transactions? Let’s analyze it in more detail.
What properties can Redis’ transaction mechanism guarantee? #
Atomicity is the most important property of transaction operations, so let’s first analyze whether Redis’ transaction mechanism can guarantee atomicity.
Atomicity #
If the transaction is executed normally without any errors, using the MULTI and EXEC commands can ensure that multiple operations are completed. However, can atomicity still be guaranteed if an error occurs during the execution of the transaction? We need to consider three scenarios.
The first scenario is when the operation command sent by the client before executing the EXEC command itself contains an error (such as a syntax error or using a nonexistent command), which is detected by the Redis instance during command queuing.
For this scenario, when the command is queued, Redis will report an error and record it. At this point, we can continue to submit command operations. However, after executing the EXEC command, Redis will refuse to execute all submitted command operations and return a transaction failed result. As a result, none of the commands in the transaction will be executed, ensuring atomicity.
Let’s take a look at a small example where the transaction fails due to an error during command queuing.
# Start the transaction
127.0.0.1:6379> MULTI
OK
# Send the first operation in the transaction, but Redis does not support the command and returns an error message
127.0.0.1:6379> PUT a:stock 5
(error) ERR unknown command `PUT`, with args beginning with: `a:stock`, `5`,
# Send the second operation in the transaction, this operation is a correct command, and Redis queues the command
127.0.0.1:6379> DECR b:stock
QUEUED
# Actually execute the transaction, but the previous command has an error, so Redis refuses to execute it
127.0.0.1:6379> EXEC
(error) EXECABORT Transaction discarded because of previous errors.
In this example, the transaction contains a PUT command that Redis does not support, so Redis reports an error when the PUT command is queued. Although there is a correct DECR command in the transaction, after executing the EXEC command, the entire transaction is aborted.
Let’s move on to the second scenario.
Different from the first scenario, this scenario is when the command and the data type of the operation do not match during command queuing, but the Redis instance does not detect the error. However, when executing the EXEC command after queuing, Redis will report an error when actually executing these transaction operations. However, it should be noted that although Redis reports errors for incorrect commands, it will still execute the correct commands. In this case, the atomicity of the transaction cannot be guaranteed.
Here’s a small example. The LPOP command in the transaction operates on a String type data, and no error is reported when it is queued, but an error is reported when EXEC is executed. The LPOP command itself was not successful, but the DECR command in the transaction was successfully executed.
# Start the transaction
127.0.0.1:6379> MULTI
OK
# Send the first operation in the transaction, the LPOP command does not match the data type, but no error is reported at this time
127.0.0.1:6379> LPOP a:stock
QUEUED
# Send the second operation in the transaction
127.0.0.1:6379> DECR b:stock
QUEUED
# Actually execute the transaction, the first operation in the transaction fails
127.0.0.1:6379> EXEC
1) (error) WRONGTYPE Operation against a key holding the wrong kind of value
2) (integer) 8
At this point, you may have a question. In traditional databases (such as MySQL), when executing a transaction, a rollback mechanism is provided. When an error occurs during transaction execution, all operations in the transaction will be rolled back and the modified data will be restored to the state before the transaction was executed. In the example we just looked at, if an error occurred during the actual execution of the command, can we use the rollback mechanism to restore the original data?
In fact, Redis does not provide a rollback mechanism. Although Redis provides the DISCARD command, this command can only be used to voluntarily abandon transaction execution and clear the command queue, it does not achieve the effect of rollback.
How should we specifically use the DISCARD command? Let’s take a look at the following code.
# Read the value of a:stock, which is 4
127.0.0.1:6379> GET a:stock
"4"
# Start the transaction
127.0.0.1:6379> MULTI
OK
# Send the first operation in the transaction, decrementing a:stock
127.0.0.1:6379> DECR a:stock
QUEUED
# Execute the DISCARD command to voluntarily abandon the transaction
127.0.0.1:6379> DISCARD
OK
# Read the value of a:stock again, the value has not been modified
127.0.0.1:6379> GET a:stock
"4"
In this example, the value of the a:stock key is initially 4. Then, we execute a transaction, trying to decrement the value of a:stock by 1. But at the end of the transaction, we execute the DISCARD command, so the transaction is abandoned. If we check the value of a:stock again, we will find that it is still 4.
Finally, let’s take a look at the third scenario: If a Redis instance fails when executing the EXEC command, causing the transaction to fail.
In this scenario, if Redis has AOF logging enabled, only part of the transaction operations will be recorded in the AOF log. We need to use the redis-check-aof tool to check the AOF log file, which can remove the unfinished transaction operations from the AOF file. This way, when we recover the instance using AOF, the transaction operations will not be executed again, thus ensuring atomicity.
Of course, if AOF logging is not enabled, when the instance is restarted, the data cannot be recovered, and in this case, atomicity is not guaranteed. Okay, up to this point, you have understood the guarantee of Redis’s transaction atomicity. Let’s summarize briefly:
- If an error occurs when a command is enqueued, the transaction execution will be abandoned, ensuring atomicity.
- If no error occurs when a command is enqueued but an error occurs during actual execution, atomicity is not guaranteed.
- If an instance failure occurs during the execution of the EXEC command and AOF log is enabled, atomicity can be guaranteed.
Next, let’s learn about the guarantee of consistency.
Consistency #
The consistency guarantee of transactions can be affected by erroneous commands and instance failures. Therefore, let’s consider the three cases based on the timing of command errors and instance failures.
Case 1: Errors occur when the command is enqueued
In this case, the transaction itself will be abandoned, ensuring the consistency of the database.
Case 2: No errors occur when the command is enqueued, but errors occur during actual execution
In this case, the erroneous commands will not be executed, and the correct commands can be executed normally without changing the consistency of the database.
Case 3: Instance failure during the execution of the EXEC command
In this case, the instance is restarted after the failure, which depends on the method of data recovery. We need to discuss the situations based on whether RDB or AOF is enabled.
If we have not enabled RDB or AOF, then the data will be lost after the instance restarts, and the database will be consistent.
If we use RDB snapshots, the transaction commands’ results are not saved in the RDB snapshot because RDB snapshots are not executed during transaction execution. Therefore, when recovering with RDB snapshots, the data in the database will still be consistent.
If we use AOF logs and the instance fails before the transaction operations are recorded in the AOF log, recovering the database using AOF logs will ensure consistency. If only part of the operations have been recorded in the AOF log, we can use redis-check-aof to remove the completed operations in the transaction, and the database will be consistent after recovery.
In summary, in the event of command execution errors or Redis failures, Redis’s transaction mechanism guarantees consistency. Next, let’s continue analyzing isolation.
Isolation #
The guarantee of transaction isolation is affected by concurrent operations executed together with the transaction. Transaction execution can be divided into two stages: command enqueueing (before the EXEC command is executed) and command actual execution (after the EXEC command is executed). Therefore, we analyze the two cases separately:
- Concurrent operations are executed before the EXEC command. To guarantee isolation, the WATCH mechanism must be used; otherwise, isolation cannot be ensured.
- Concurrent operations are executed after the EXEC command. In this case, isolation can be guaranteed. Let’s start with the first case. When the EXEC command of a transaction has not been executed yet, the transaction commands are temporarily stored in a command queue. At this point, if there are other concurrent operations, we need to check whether the transaction is using the WATCH mechanism.
The purpose of the WATCH mechanism is to monitor the value changes of one or more keys before executing the transaction. When the transaction calls the EXEC command to execute, the WATCH mechanism will first check whether the monitored keys have been modified by other clients. If they have been modified, the transaction execution will be abandoned to avoid compromising the isolation of the transaction. Then, the client can execute the transaction again. If there are no concurrent operations modifying the transaction data, the transaction can be executed normally, and the isolation is also guaranteed.
The specific implementation of the WATCH mechanism is achieved by the WATCH command. Let me give you an example. You can take a look at the diagram below to further understand the usage of the WATCH command.
Let me explain the content shown in the diagram.
At time t1, client X sends a WATCH command to the instance. After receiving the WATCH command, the instance starts monitoring the changes in the value of a:stock.
Then, at time t2, client X sends the MULTI command and DECR command to the instance, and the DECR command is temporarily stored in the command queue.
At time t3, client Y also sends a DECR command to the instance to modify the value of a:stock. The instance executes the command directly upon receiving it.
By time t4, when the instance receives the EXEC command sent by client X, the WATCH mechanism detects that a:stock has been modified, so it abandons the execution of the transaction. In this way, the isolation of the transaction can be guaranteed.
Of course, without using the WATCH mechanism, concurrent operations executed before the EXEC command will read and write data. And when executing the EXEC command, the data that the transaction needs to operate has already changed. In this case, Redis does not ensure that the transaction is isolated from other operations, and the isolation is not guaranteed. The following diagram shows the situation without the WATCH mechanism, you can take a look.
At time t2, the EXEC command sent by client X has not been executed yet, but the DECR command of client Y has been executed. At this time, the value of a:stock will be modified, and the isolation of the transaction initiated by client X cannot be guaranteed.
We have just discussed the situation where concurrent operations are executed before the EXEC command. Now, let me talk about the second case: concurrent operations are received and executed by the server after the EXEC command.
Because Redis executes commands in a single thread, and after the EXEC command is executed, Redis ensures that all the commands in the command queue are executed first. Therefore, in this case, concurrent operations will not compromise the isolation of the transaction, as shown in the following diagram:
Finally, let’s analyze the guarantee of the durability property of Redis transactions.
Durability #
Because Redis is an in-memory database, whether the data is persistently stored depends entirely on the Redis’s persistence configuration mode.
If Redis does not use RDB or AOF, then the durability property of the transaction is definitely not guaranteed. If Redis uses the RDB mode, then after a transaction is executed and before the next RDB snapshot is executed, if the instance crashes, the data modified by the transaction cannot be guaranteed to be persisted.
If Redis adopts the AOF mode, there are data loss situations for all three configuration options: no, everysec, and always. Therefore, the durability property of the transaction is still not guaranteed.
Therefore, regardless of the persistence mode Redis adopts, the durability property of the transaction cannot be guaranteed.
Summary #
In this lesson, we learned about the implementation of transactions in Redis. Redis supports the transaction mechanism through four commands: MULTI, EXEC, DISCARD, and WATCH. The functions of these four commands are summarized in the table below, which you can review again.
The ACID properties of transactions are the basic requirements for using transactions correctly. Through the analysis in this lesson, we learned that Redis’ transaction mechanism can guarantee consistency and isolation, but it cannot guarantee durability. However, since Redis is an in-memory database, durability is not a necessary property. We are more concerned about the atomicity, consistency, and isolation of transactions.
The situation regarding atomicity is more complex. Atomicity is only not guaranteed when there is a syntax error in the commands used in the transaction. In other cases, transactions can be executed atomically.
Therefore, I have a small suggestion for you: Develop your programs strictly according to the command specifications of Redis, and ensure the correctness of the commands through code review. This way, the transaction mechanism of Redis can be applied in practice to ensure the correct execution of multiple operations.
One question per lesson #
As usual, I have a small question for you. If a Redis instance fails during the execution of a transaction, and Redis uses the RDB mechanism, can the atomicity of the transaction be guaranteed?
Feel free to write down your thoughts and answers in the comments. Let’s discuss together. If you find today’s content helpful, you are also welcome to share it with your friends or colleagues. See you in the next lesson.