36 Practical Redis Master Slave Synchronization

36 Practical Redis Master-Slave Synchronization #

Master-slave replication is the cornerstone of Redis high availability services and the most fundamental aspect of multi-server operations. We refer to the main node that stores the primary data as the master node, and the replica nodes that replicate data from the master node as slave nodes, as shown in the diagram below:

Master-Slave Replication.png

In Redis, a master node can have multiple slave nodes, and a slave node can also be a master node for other servers, as shown in the diagram below:

Master-Slave Replication - Slave-Slave Mode.png

Advantages of Master-Slave Replication #

Master-slave replication has the following three advantages:

Performance: With master-slave replication, query tasks can be assigned to slave servers, while write operations are performed by the master server. This greatly improves the efficiency of program execution by distributing the load across multiple servers.
High availability: With master-slave replication, when the master server node goes down, a slave node can be quickly promoted to become the new master node. This saves valuable time for recovering from Redis server failures.
Data loss prevention: When the disk of the master server fails, the other slave servers still retain the relevant data, preventing complete data loss.

Since master-slave replication has so many advantages, let’s now see how to enable and use this feature.

Enabling Master-Slave Replication #

Setting up a slave server during runtime #

During the runtime of Redis, we can use the replicaof host port command to set ourselves as a slave server of the target IP. Execute the following command:

127.0.0.1:6379> replicaof 127.0.0.1 6380
OK

If the master server has a password set, you need to enter the password of the master server on the slave server using the config set masterauth master_password command, for example:

127.0.0.1:6377> config set masterauth pwd654321
OK

1. Execution process

After executing the replicaof command, the data on the slave server will be cleared, and the master server will synchronize its data to the slave server.

2. Testing the replication functionality

After setting up the master and slave servers for replication, let’s test the replication of data. First, we will execute a command to store data on the master server, and then query it from the slave server.

Command executed on the master server:

127.0.0.1:6379> set lang redis
OK

Query executed on the slave server:

127.0.0.1:6379> get lang
"redis"

As we can see, the data has been successfully replicated.

Setting up a slave server during startup #

We can use the command redis-server --port 6380 --replicaof 127.0.0.1 6379 to set ourselves as a slave server of the target server.

Data Replication #

Full data replication #

When a new slave server connects, to ensure the consistency of multiple databases, the master server will execute the bgsave command to generate an RDB file, and then send it to the slave server over a socket. After receiving the RDB file, the slave server loads all the data into its own program, completing a full data replication.

Partial data replication #

Before Redis 2.8, whenever a slave server went offline and came back online, the master server would perform a complete data replication. However, this approach is inefficient and wasteful when only a small amount of data needs to be replicated in cases where the offline time is short. Redis 2.8 optimized this scenario.

In Redis 2.8, when a slave server goes offline, the master server stores the write commands that occurred during the offline period in a queue of a specific size. The queue ensures the execution order is first in, first out. When the slave server recovers and comes back online, the master server checks if the commands from the offline period are still in the queue. If they are, the master server directly sends the data from the queue to the slave server, thus avoiding the waste of resources caused by full replication.

Tip: The default size of the queue for storing offline commands is 1MB. Users can modify the configuration item repl-backlog-size to adjust the size of the queue.

Diskless replication #

From the previous content, we learned that during the initial connection between the master and slave, an RDB file is first generated and then sent to the slave server. If the master server has a non-solid-state disk, the I/O operations on the disk can be very high. To alleviate this problem, Redis 2.8.18 introduced diskless replication. With diskless replication, an RDB file is not created locally. Instead, a subprocess is spawned, and the subprocess directly writes the RDB file to the slave server over a socket. This allows the master server to complete data replication with the slave server without creating an RDB file. To use the diskless synchronization feature, simply set the value of the configuration option repl-diskless-sync to yes. Its default value is no.

Querying the Server’s Role #

We can use the role command to query the current server’s master-slave role information.

Viewing the Master Server’s Role #

Executing the role command on the master server will yield the following result:

127.0.0.1:6379> role
1) "master"
2) (integer) 546
3) 1) 1) "172.17.0.1"
      2) "6379"
      3) "546"

“master” indicates that it is the master server, followed by the IP, port, and connection time of the slave servers.

Viewing the Slave Server’s Role #

Executing the role command on the slave server will yield the following result:

127.0.0.1:6379> role
1) "slave"
2) "192.168.1.71"
3) (integer) 6380
4) "connected"
5) (integer) 14

“slave” indicates that it is the slave server, followed by the IP, port, and connection time of the master server.

Disabling Master-Slave Synchronization #

We can use the replicaof no one command to stop replication on the slave server. The following commands demonstrate this operation:

127.0.0.1:6379> role # Querying the current role
1) "slave" # Slave server
2) "192.168.1.71"
3) (integer) 6380
4) "connected"
5) (integer) 14
127.0.0.1:6379> replicaof no one # Disabling synchronization
OK
127.0.0.1:6379> role # Querying the current role
1) "master" # Master server
2) (integer) 1097
3) (empty list or set)

After executing the replicaof no one command, the slave server becomes the master server.

Note: The transformation of server types does not affect the data, and the data on this server will be preserved.

Things to Note #

There are some points to note regarding master-slave synchronization.

Data Consistency Issue #

After the slave server has completed the data synchronization with the master server, newly added commands will be sent to the slave server asynchronously. During this process, there may be a short period of data inconsistency in the master-slave synchronization. For example, if the master server crashes before this asynchronous synchronization occurs, it will lead to data inconsistency.

Read-Only on the Slave Server #

By default, the master server in replication mode can perform both write and read operations, while the slave server can only perform read operations.

The config set replica-read-only no command can be executed on the slave server to enable write mode. However, the following points should be noted:

The data written on the slave server will not be synchronized to the master server.
When the data on the master server is the same as the data on the slave server, the data on the master server will overwrite the data on the slave server.
The data on the slave server will be cleared during a full data synchronization.

Changes in Replication Commands #

Before Redis 5.0, the replication command used was slaveof. Starting from Redis 5.0, the replication command was changed to replicaof. In higher versions (Redis 5+), replicaof should be used as much as possible, as the slaveof command may be deprecated at any time.

Conclusion #

In this article, we have learned about the basic functionality of master-slave synchronization in Redis distributed deployment. Master-slave synchronization can be enabled using the replicaof host port command. We have also learned about the three synchronization methods: full data synchronization (initial full RDB synchronization), partial data synchronization (Redis 2.8 optimization for short-term offline synchronization), and diskless synchronization (synchronization of data not generated by RDB). We can also use the replicaof no one command to stop the replication on the slave server.