24 RocketMQConsole Common Pages Index Retrieval Logic #
The purpose of this article is not to provide a detailed introduction to the usage of RocketMQ-Console, but to introduce some key points (mostly points that may cause confusion) and avoid making assumptions about the returned results.
Cluster Overview #
Cluster provides an overview of all the broker information in a cluster, including the master and slave nodes. Sometimes, there may be some discrepancies in the statistics between the master and slave nodes. For example, the Today Producer Count of a slave node may be lower or higher than that of the master node. In such cases, it is not necessary to pay too much attention or misunderstand the results. Here’s why:
RocketMQ’s data statistics are based on time windows and the data is stored in memory. Once a broker node is restarted, all monitoring data will be lost. Additionally, there is a delay in synchronizing data between the master and slave nodes, so it is normal to have inconsistent statistics. If the master broker node has been restarted, the statistics data will be less than that of the slave node. Similarly, if the slave broker node has been restarted, the statistics data of the master node will exceed that of the slave node.
Consumption TPS Only Counts the Master Node #
In the Consumer menu, the displayed TPS represents the message consumption TPS. However, it is important to note that the data only comes from the master node and does not include the statistics of the slave nodes. I once encountered a problem with a consumer where I needed to reset the consumer’s offset to a few days ago. The result was that the delay (message backlog) decreased, but the TPS remained 0. This happened because after resetting the offset, the consumption shifted to the slave node, and the consumption TPS on the slave node was not counted.
RocketMQ msgId #
In RocketMQ, there are two message IDs: offsetMsgId (which records the physical offset and other information) and msgId (which is the globally unique ID of the message). So, which ID is used for the messages in the displayed list? The message ID shown here is the msgId.
When searching for a message based on the MESSAGE ID, the following interface can be used to search by msgId or offsetMsgId. The interface looks like this:
Why is it like this? It’s because RocketMQ-Console is compatible and will first try to query based on offsetMsgId. If the query fails, it will then use msgId for a quick search. It is not surprising that messages can be quickly queried based on offsetMsgId, as offsetMsgId contains the broker’s IP address, port number, and physical offset. But how can messages be quickly retrieved based on msgId? The answer is through a hash index. The globally unique ID, also known as UNIQ_KEY, which is msgId in RocketMQ, is stored in the index file.
Creating Subscription Relationships #
In RocketMQ, you can not only disable automatic topic creation, but also disable automatic consumer group creation. By setting the attribute autoCreateSubscriptionGroup to false, automatic consumer group creation can be disabled. In this case, consumer groups must be manually created through commands or the interface before the project team can use them to consume messages. It is recommended to set this attribute to false in a production environment for better control. Consumer group subscription information can be added through this interface in RocketMQ:
Understanding the lastTimestamp Time in RocketMQ-Console #
You may typically see the time of lastTimestamp displayed as 1970. Why is this?
Firstly, in the “View Message Consumption Progress” section, lastTimestamp represents the storage time of the currently consumed message, which means the storage time of the message corresponding to the current offset in the broker. In combination with RocketMQ’s message consumption expired deletion mechanism, which stores a message for only three days by default, if a message has not been consumed within three days, it will be deleted. If the message corresponding to the current offset in the consumption progress has been deleted, it will display 1970.
Using RocketMQ is relatively simple. This article primarily focuses on some points that are prone to “misunderstandings”. If you have any other questions about using RocketMQ-Console, please feel free to join the official WeChat group.