01 Setting Up Learning Environment Preparation

01 Setting up Learning Environment - Preparation #

This article is divided into the following parts:

Installing RocketMQ and RocketMQ-Console on a Linux server
Setting up a debuggable environment in IDEA

Installing RocketMQ and RocketMQ-Console on Linux #

Installing RocketMQ #

Step 1: Download the RocketMQ package from the following link

cd /opt/application
wget https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/4.7.1/rocketmq-all-4.7.1-bin-release.zip

Step 2: Extract the package

unzip rocketmq-all-4.7.1-bin-release.zip
ls -l

The extracted files will be shown as follows:

The conf folder contains the configuration files for RocketMQ, providing example configurations for various deployment structures. For example, 2m-2s-async is an example configuration for a 2 master 2 slave asynchronous replication; 2m-noslave is an example configuration for 2 masters. Since this article is about setting up a learning environment, we will use a deployment structure with 1 master. How to set up a RocketMQ cluster and optimize parameters in a production environment will be covered in future articles in this series.

Step 3: Modify the NameServer JVM parameters

cd bin
vi runserver.sh

# Locate the following code
JAVA_OPT="${JAVA_OPT} -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

# Modify the "-Xms -Xmx -Xmn" parameters
JAVA_OPT="${JAVA_OPT} -server -Xms512M -Xmx512M -Xmn256M -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

Note: The purpose of modifying the JVM parameters here is because my personal learning computer does not have enough memory. The default NameServer would consume 4G.

Step 4: Start the NameServer

nohup ./mqnamesrv &

Check the ${user_home}/logs/rocketmqlogs/namesrv.log log file. If the output is as shown in the following image, it means the NameServer has been successfully started.

Step 5: Modify the Broker configuration file

vi conf/broker.conf
# Use the following configuration file
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
storePathRootDir=/data/rocketmq/store
storePathCommitLog=/data/rocketmq/store/commitlog
namesrvAddr=127.0.0.1:9876
brokerIP1=192.168.3.10
brokerIP2=192.168.3.10
autoCreateTopicEnable=false

Step 6: Modify the Broker JVM parameters

cd bin
vi runbroker.sh 
# Modify the following configuration (before modification)
JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"
# After modification
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"

Step 7: Start the Broker

cd bin
nohup ./mqbroker -c ../conf/broker.conf &

Check the ${user_home}/logs/rocketmqlogs/broker.log file. If the output is as shown in the following image, it means the Broker has been successfully started.

After following the above steps, you have successfully installed RocketMQ NameServer and Broker servers on a Linux environment.

Note: If an error occurs during the installation process, you can check the logs in ${user_home}/logs/rocketmqlogs to determine the cause of the error. ${user_home} refers to the user’s home directory.

In this directory, there will be many log files. If you are not familiar with the meanings of these files, you can use the ls -l command to view the files with a non-zero file size one by one, in order to find the error log and quickly resolve the issue.

RocketMQ provides many operational commands to view the running status of the RocketMQ cluster. Here, I will use the clusterList command to simply view the cluster status and verify its status.

sh ./mqadmin clusterList -n 127.0.0.1:9876

The result of running this command is shown in the following image:

Installing RocketMQ-Console #

Using operational commands is not very intuitive and has a high learning curve. Therefore, RocketMQ provides a web-based management interface called RocketMQ-Console, which provides commonly used operational functions for RocketMQ clusters. This section explains how to install RocketMQ-Console on a Linux environment.

RocketMQ does not provide an installation package for RocketMQ-Console, so it needs to be compiled from source code.

Step 1: Download the source code

wget https://github.com/apache/rocketmq-externals/archive/rocketmq-console-1.0.0.tar.gz
tar -xf rocketmq-console-1.0.0.tar.gz
# Rename for convenience
mv rocketmq-externals-rocketmq-console-1.0.0/rocketmq-console  rocketmq-console

Step 2: Modify the configuration file

cd rocketmq-console
vi src/main/resources/applications.properties

The main modification is to change the NameServer address that is being pointed to. The modified result is shown in the following figure:

Step3: Compile the source code using Maven command

mvn clean package -DskipTests

After compilation, a runnable jar package will be generated in the target directory, as shown in the following figure:

Step4: Copy the package to your preferred software installation directory, for example, I like to put it in /opt/application/

cp rocketmq-console-ng-1.0.0.jar /opt/application/

Step5: Start RocketMQ-Console

nohup java -jar rocketmq-console-ng-1.0.0.jar &

Enter http://localhost:8080 in the browser to see if the installation is successful. If the following figure appears, it means the installation was successful.

Exception Analysis and Troubleshooting #

If unexpected errors occur during the installation process, don’t panic. By checking the relevant log files and looking for error logs, you can think about or search for solutions on Baidu. I believe you can easily resolve them.

For example, if RocketMQ and RocketMQ-Console are started using the Baseuser, the relevant log paths are as follows:

RocketMQ: /home/baseuser/logs/rocketmqlogs/
RocketMQ-Console: /home/baseuser/logs/consolelogs

Installing RocketMQ in IDEA #

Most programmers trust the debugging tools when it comes to development and debugging. Can we debug the RocketMQ source code in IDEA? The answer is yes. This section will demonstrate how to run the RocketMQ NameServer and Broker components in IDEA and perform debugging.

Step1: Download the RocketMQ source code from GitHub and import it into IDEA

The screenshot is as follows:

Step2: Set the environment variable ROCKETMQ_HOME in namesrv/src/main/java/org/apache/rocketmq/namesrv/NamesrvStartup

The steps are shown in the following figure:

Set the environment variable name: ROCKETMQ_HOME, and its value specifies the main directory where RocketMQ runs. The path I set is: /home/dingwpmz/tmp/rocketmq.

Step3: Copy the distribution/conf/logback_namesrv.xml file to the main directory set in Step 2

The result after execution is shown in the following figure:

Tips: This file is the log path of NameServer. You can manually modify the log directory in the logback_namesrv.xml file. Since this is basic knowledge of Logback, I will not elaborate on the configuration method here.

Step4: Run NamesrvStartup in Debug mode. The execution result indicates successful startup

Step5: Copy the distribution/conf/logback_broker.xml and broker.conf files to the main directory set in Step 2

The result after execution is shown in the following figure:

Step6: Modify the configuration in broker.conf, mainly setting the address of NameServer, the name of the Broker, and other related attributes

vi broker.conf
# Use the following configuration file
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
storePathRootDir=/home/dingwpmz/tmp/rocketmq/store
storePathCommitLog=/home/dingwpmz/tmp/rocketmq/store/commitlog
namesrvAddr=127.0.0.1:9876
brokerIP1=192.168.3.10
brokerIP2=192.168.3.10
autoCreateTopicEnable=true

Step7: Set the environment variable ROCKETMQ_HOME in broker/src/main/java/org/apache/rocketmq/broker/BrokerStartup

The steps are shown in the following figure:

Step8: Run BrokerStartup in Debug mode

The result is shown in the following figure:

Seeing this prompt means we have succeeded.

Next, let’s do a simple verification.

First, set a breakpoint in the parseRequestHeader method of the AbstractSendMessageProcessor class.

Then run the org/apache/rocketmq/example/quickstart/Producer in the example package to see if it enters the breakpoint. The result is shown in the following figure, indicating that it has entered Debug mode.

Summary #

As the first article in the RocketMQ practical series, the purpose of this article is to build a learning environment for studying RocketMQ. So it is developed in two aspects:

Installing RocketMQ and RocketMQ-Console in a Linux environment.
Running RocketMQ in IDEA to build an environment where RocketMQ can be debugged.

Tips: Building a debuggable environment does not mean studying the RocketMQ source code. We should only use debugging when we cannot understand a certain piece of code. By leveraging some runtime data, it becomes easier to understand.