16 Rocket Mq Cluster Performance Tuning

16 RocketMQ Cluster Performance Tuning #

Introduction #

When setting up a cluster in a production environment, it’s necessary to boost the performance of the cluster. In other words, we need to have an idea of the cluster’s maximum TPS (Transactions Per Second). Typically, we control the actual traffic in the range of 1/3 to 1/2 of the maximum value obtained from stress testing, leaving one to two times the capacity to handle traffic surges.

How to perform stress testing?

  1. Write some code to send messages for testing. This can be done by using JMeter for stress testing or by sending messages through multiple threads in the code. This method requires multiple well-configured testing machines.
  2. Use the built-in stress test script provided by RocketMQ.

Both methods have been used in practice, and the stress testing results are basically similar. To make it convenient, it is recommended to directly use the stress test script on the newly established RocketMQ cluster.

Stress Test Script #

After extracting the RocketMQ installation package, there is a producer.sh file in the benchmark directory. We can use this script to perform stress testing.

Let’s take a look at the meaning of each field using producer.sh -h.

Fields and their meanings:

Name Meaning
-h Help
-k Whether the messages have keys during testing. The default is false.
-n NameServer address.
-s Message size. The default is 128 bytes.
-t Topic name.
-w Number of concurrent threads. The default is 64.

Boosting Performance in Practice #

The system configuration is 48C256G, and the cluster architecture consists of 4 masters and 4 slaves. We will perform tests on this cluster in different scenarios and observe the output results. You can flexibly combine the scenarios according to the actual situation, and different combinations will produce different results. However, the stress testing method remains the same.

Test Scenario 1 #

1 thread, message size of 1K, 8 queues in the topic. The highest TPS achieved among the results is 4533, the maximum RT (Response Time) is 299, and the average RT is 0.22.

sh producer.sh -t cluster-perf-tst8 -w 1 -s 1024 -n x.x.x.x:9876
Send TPS: 4281 Max RT: 299 Average RT:   0.233 Send Failed: 0 Response Failed: 0
Send TPS: 4237 Max RT: 299 Average RT:   0.236 Send Failed: 0 Response Failed: 0
Send TPS: 4533 Max RT: 299 Average RT:   0.221 Send Failed: 0 Response Failed: 0
Send TPS: 4404 Max RT: 299 Average RT:   0.227 Send Failed: 0 Response Failed: 0
Send TPS: 4360 Max RT: 299 Average RT:   0.229 Send Failed: 0 Response Failed: 0
Send TPS: 4269 Max RT: 299 Average RT:   0.234 Send Failed: 0 Response Failed: 0
Send TPS: 4319 Max RT: 299 Average RT:   0.231 Send Failed: 0 Response Failed: 0

Test Scenario 2 #

1 thread, message size of 3K, 8 queues in the topic. The highest TPS achieved among the results is 4125, the maximum RT is 255, and the average RT is 0.24.

sh producer.sh -t cluster-perf-tst8 -w 1 -s 3072 -n 192.168.x.x:9876
Send TPS: 4120 Max RT: 255 Average RT:   0.242 Send Failed: 0 Response Failed: 0
Send TPS: 4054 Max RT: 255 Average RT:   0.246 Send Failed: 0 Response Failed: 0
Send TPS: 4010 Max RT: 255 Average RT:   0.249 Send Failed: 0 Response Failed: 0
Send TPS: 4125 Max RT: 255 Average RT:   0.242 Send Failed: 0 Response Failed: 0
Send TPS: 4093 Max RT: 255 Average RT:   0.244 Send Failed: 0 Response Failed: 0
Send TPS: 4093 Max RT: 255 Average RT:   0.244 Send Failed: 0 Response Failed: 0
Send TPS: 3999 Max RT: 255 Average RT:   0.250 Send Failed: 0 Response Failed: 0
Send TPS: 3957 Max RT: 255 Average RT:   0.253 Send Failed: 0 Response Failed: 0

Test Scenario 3 #

1 thread, message size of 1K, 16 queues in the topic. The highest TPS achieved among the results is 5289, the maximum RT is 255, and the average RT is 0.19.

sh producer.sh -t cluster-perf-tst16 -w 1 -s 1024 -n x.x.x.x:9876
Send TPS: 5289 Max RT: 225 Average RT:   0.189 Send Failed: 0 Response Failed: 0
Send TPS: 5252 Max RT: 225 Average RT:   0.190 Send Failed: 0 Response Failed: 0
Send TPS: 5124 Max RT: 225 Average RT:   0.195 Send Failed: 0 Response Failed: 0
Send TPS: 5146 Max RT: 225 Average RT:   0.194 Send Failed: 0 Response Failed: 0

1个线程，消息大小为3K，主题为16个队列。以下结果中发送最大TPS为5011，最大RT为244，平均RT为0.21。

sh producer.sh -t cluster-perf-tst16 -w 1 -s 3072 -n x.x.x.x:9876
Send TPS: 4778 Max RT: 244 Average RT:   0.209 Send Failed: 0 Response Failed: 0
Send TPS: 5011 Max RT: 244 Average RT:   0.199 Send Failed: 0 Response Failed: 0
Send TPS: 4826 Max RT: 244 Average RT:   0.207 Send Failed: 0 Response Failed: 0
Send TPS: 4762 Max RT: 244 Average RT:   0.210 Send Failed: 0 Response Failed: 0
Send TPS: 4663 Max RT: 244 Average RT:   0.214 Send Failed: 0 Response Failed: 0
Send TPS: 4648 Max RT: 244 Average RT:   0.215 Send Failed: 0 Response Failed: 0
Send TPS: 4778 Max RT: 244 Average RT:   0.209 Send Failed: 0 Response Failed: 0
Send TPS: 4737 Max RT: 244 Average RT:   0.211 Send Failed: 0 Response Failed: 0
Send TPS: 4523 Max RT: 244 Average RT:   0.221 Send Failed: 0 Response Failed: 0
Send TPS: 4544 Max RT: 244 Average RT:   0.220 Send Failed: 0 Response Failed: 0
Send TPS: 4683 Max RT: 244 Average RT:   0.213 Send Failed: 0 Response Failed: 0
Send TPS: 4838 Max RT: 244 Average RT:   0.207 Send Failed: 0 Response Failed: 0

10 threads, message size 3K, 16 queues. The maximum TPS sent in the following results is 39976, the maximum RT is 237, and the average RT is 0.25.

sh producer.sh -t cluster-perf-tst16 -w 10 -s 3072 -n x.x.x.x:9876
Send TPS: 36245 Max RT: 237 Average RT:   0.276 Send Failed: 0 Response Failed: 0
Send TPS: 38713 Max RT: 237 Average RT:   0.258 Send Failed: 0 Response Failed: 0
Send TPS: 36327 Max RT: 237 Average RT:   0.275 Send Failed: 0 Response Failed: 0
Send TPS: 39005 Max RT: 237 Average RT:   0.256 Send Failed: 0 Response Failed: 0
Send TPS: 37926 Max RT: 237 Average RT:   0.264 Send Failed: 0 Response Failed: 0
Send TPS: 38804 Max RT: 237 Average RT:   0.258 Send Failed: 0 Response Failed: 0
Send TPS: 39976 Max RT: 237 Average RT:   0.250 Send Failed: 0 Response Failed: 0

Test Scenario 9 #

30 threads, message size 1K, 8 queues. The maximum TPS sent in the following results is 89288, the maximum RT is 309, and the average RT is 0.34.

sh producer.sh -t cluster-perf-tst8 -w 30 -s 1024 -n x.x.x.x:9876
Send TPS: 86259 Max RT: 309 Average RT:   0.348 Send Failed: 0 Response Failed: 0
Send TPS: 85335 Max RT: 309 Average RT:   0.351 Send Failed: 0 Response Failed: 0
Send TPS: 81850 Max RT: 309 Average RT:   0.366 Send Failed: 0 Response Failed: 0
Send TPS: 87712 Max RT: 309 Average RT:   0.342 Send Failed: 0 Response Failed: 0
Send TPS: 89288 Max RT: 309 Average RT:   0.336 Send Failed: 0 Response Failed: 0
Send TPS: 86732 Max RT: 309 Average RT:   0.346 Send Failed: 0 Response Failed: 0

Test Scenario 10 #

30 threads, message size 3K, 8 queues. The maximum TPS sent in the following results is 77792, the maximum RT is 334, and the average RT is 0.42.

sh producer.sh -t cluster-perf-tst8 -w 30 -s 3072 -n x.x.x.x:9876
Send TPS: 74085 Max RT: 334 Average RT:   0.405 Send Failed: 0 Response Failed: 0
Send TPS: 71014 Max RT: 334 Average RT:   0.422 Send Failed: 0 Response Failed: 0
Send TPS: 77792 Max RT: 334 Average RT:   0.386 Send Failed: 0 Response Failed: 0
Send TPS: 73913 Max RT: 334 Average RT:   0.406 Send Failed: 0 Response Failed: 0
Send TPS: 77337 Max RT: 334 Average RT:   0.392 Send Failed: 0 Response Failed: 0
Send TPS: 72184 Max RT: 334 Average RT:   0.416 Send Failed: 0 Response Failed: 0
Send TPS: 77271 Max RT: 334 Average RT:   0.388 Send Failed: 0 Response Failed: 0
Send TPS: 75016 Max RT: 334 Average RT:   0.400 Send Failed: 0 Response Failed: 0

Test Scenario 11 #

30 threads, message size 1K, 16 queues. The maximum TPS sent in the following results is 87009, the maximum RT is 306, and the average RT is 0.34.

sh producer.sh -t zms-clusterB-perf-tst16 -w 30 -s 1024 -n x.x.x.x:9876
Send TPS: 82946 Max RT: 306 Average RT:   0.362 Send Failed: 0 Response Failed: 0
Send TPS: 86902 Max RT: 306 Average RT:   0.345 Send Failed: 0 Response Failed: 0
Send TPS: 83157 Max RT: 306 Average RT:   0.365 Send Failed: 0 Response Failed: 0
Send TPS: 86804 Max RT: 306 Average RT:   0.345 Send Failed: 0 Response Failed: 0
Send TPS: 87009 Max RT: 306 Average RT:   0.345 Send Failed: 0 Response Failed: 0
Send TPS: 80219 Max RT: 306 Average RT:   0.374 Send Failed: 0 Response Failed: 0

Test Scenario 12 #

30 threads, message size 3K, 16 queues. The maximum TPS sent in the following results is 78555, the maximum RT is 329, and the average RT is 0.40.

sh producer.sh -t cluster-perf-tst16 -w 30 -s 3072 -n x.x.x.x:9876
Send TPS: 73864 Max RT: 329 Average RT:   0.403 Send Failed: 0 Response Failed: 0
Send TPS: 78555 Max RT: 329 Average RT:   0.382 Send Failed: 0 Response Failed: 0
Send TPS: 75200 Max RT: 329 Average RT:   0.406 Send Failed: 0 Response Failed: 0
Send TPS: 73925 Max RT: 329 Average RT:   0.406 Send Failed: 0 Response Failed: 0
Send TPS: 69955 Max RT: 329 Average RT:   0.429 Send Failed: 0 Response Failed: 0

Test Scenario 13 #

In test scenario 14, there were 45 threads, the message size was 3K, and there were 8 queues. The maximum TPS for sending was 90403, the maximum RT was 462, and the average RT was 0.52.

In test scenario 15, there were 45 threads, the message size was 1K, and there were 16 queues. The maximum TPS for sending was 100158, the maximum RT was 604, and the average RT was 0.49.

In test scenario 16, there were 45 threads, the message size was 3K, and there were 16 queues. The maximum TPS for sending was 77297, the maximum RT was 436, and the average RT was 0.39.

In test scenario 17, there were 60 threads, the message size was 1K, and there were 8 queues. The maximum TPS for sending was 111395, the maximum RT was 369, and the average RT was 0.53. 60 threads, message size of 3K, and 8 topic queues. The maximum TPS for sending in the following results is 99535, the maximum RT is 583, and the average RT is 0.64.

sh producer.sh -t cluster-perf-tst8 -w 60 -s 3072 -n 192.168.x.x:9876
Send TPS: 92572 Max RT: 583 Average RT:   0.648 Send Failed: 0 Response Failed: 0
Send TPS: 95163 Max RT: 583 Average RT:   0.640 Send Failed: 0 Response Failed: 1
Send TPS: 93823 Max RT: 583 Average RT:   0.654 Send Failed: 0 Response Failed: 1
Send TPS: 97091 Max RT: 583 Average RT:   0.628 Send Failed: 0 Response Failed: 1
Send TPS: 98205 Max RT: 583 Average RT:   0.628 Send Failed: 0 Response Failed: 1
Send TPS: 99535 Max RT: 583 Average RT:   0.596 Send Failed: 0 Response Failed: 3

Test Scenario 19 #

60 threads, message size of 1K, and 16 topic queues. The maximum TPS for sending in the following results is 111667, the maximum RT is 358, and the average RT is 0.55.

sh producer.sh -t cluster-perf-tst16 -w 60 -s 1024 -n x.x.x.x:9876
Send TPS: 105229 Max RT: 358 Average RT:   0.578 Send Failed: 0 Response Failed: 0
Send TPS: 103003 Max RT: 358 Average RT:   0.582 Send Failed: 0 Response Failed: 0
Send TPS: 95497 Max RT: 358 Average RT:   0.628 Send Failed: 0 Response Failed: 0
Send TPS: 108878 Max RT: 358 Average RT:   0.551 Send Failed: 0 Response Failed: 0
Send TPS: 109265 Max RT: 358 Average RT:   0.549 Send Failed: 0 Response Failed: 0
Send TPS: 105545 Max RT: 358 Average RT:   0.568 Send Failed: 0 Response Failed: 0
Send TPS: 111667 Max RT: 358 Average RT:   0.537 Send Failed: 0 Response Failed: 0

Test Scenario 20 #

60 threads, message size of 3K, and 16 topic queues. The maximum TPS for sending in the following results is 101073, the maximum RT is 358, and the average RT is 0.61.

sh producer.sh -t cluster-perf-tst16 -w 60 -s 3072 -n x.x.x.x:9876
Send TPS: 98899 Max RT: 358 Average RT:   0.606 Send Failed: 0 Response Failed: 0
Send TPS: 101073 Max RT: 358 Average RT:   0.594 Send Failed: 0 Response Failed: 0
Send TPS: 97295 Max RT: 358 Average RT:   0.617 Send Failed: 0 Response Failed: 0
Send TPS: 97923 Max RT: 358 Average RT:   0.609 Send Failed: 0 Response Failed: 1
Send TPS: 96111 Max RT: 358 Average RT:   0.620 Send Failed: 0 Response Failed: 2
Send TPS: 93873 Max RT: 358 Average RT:   0.639 Send Failed: 0 Response Failed: 2
Send TPS: 96466 Max RT: 358 Average RT:   0.622 Send Failed: 0 Response Failed: 2
Send TPS: 96579 Max RT: 358 Average RT:   0.621 Send Failed: 0 Response Failed: 2

Test Scenario 21 #

75 threads, message size of 1K, and 8 topic queues. The maximum TPS for sending in the following results is 112707, the maximum RT is 384, and the average RT is 0.68.

sh producer.sh -t cluster-perf-tst8 -w 75 -s 1024 -n x.x.x.x:9876
Send TPS: 108367 Max RT: 384 Average RT:   0.692 Send Failed: 0 Response Failed: 0
Send TPS: 107516 Max RT: 384 Average RT:   0.701 Send Failed: 0 Response Failed: 0
Send TPS: 110974 Max RT: 384 Average RT:   0.680 Send Failed: 0 Response Failed: 0
Send TPS: 109754 Max RT: 384 Average RT:   0.683 Send Failed: 0 Response Failed: 0
Send TPS: 111917 Max RT: 384 Average RT:   0.670 Send Failed: 0 Response Failed: 0
Send TPS: 104764 Max RT: 384 Average RT:   0.712 Send Failed: 0 Response Failed: 1
Send TPS: 112208 Max RT: 384 Average RT:   0.668 Send Failed: 0 Response Failed: 1
Send TPS: 112707 Max RT: 384 Average RT:   0.665 Send Failed: 0 Response Failed: 1

Test Scenario 22 #

75 threads, message size of 3K, and 8 topic queues. The maximum TPS for sending in the following results is 103953, the maximum RT is 370, and the average RT is 0.74.

sh producer.sh -t cluster-perf-tst8 -w 75 -s 3072 -n x.x.x.x:9876
Send TPS: 102311 Max RT: 370 Average RT:   0.733 Send Failed: 0 Response Failed: 0
Send TPS: 93722 Max RT: 370 Average RT:   0.800 Send Failed: 0 Response Failed: 0
Send TPS: 101091 Max RT: 370 Average RT:   0.742 Send Failed: 0 Response Failed: 0
Send TPS: 100404 Max RT: 370 Average RT:   0.747 Send Failed: 0 Response Failed: 0
Send TPS: 102328 Max RT: 370 Average RT:   0.733 Send Failed: 0 Response Failed: 0
Send TPS: 103953 Max RT: 370 Average RT:   0.722 Send Failed: 0 Response Failed: 0
Send TPS: 103454 Max RT: 370 Average RT:   0.725 Send Failed: 0 Response Failed: 0

Test Scenario 23 #

75 threads, message size of 1K, 16 queues. The maximum TPS (Transactions Per Second) for sending in the following results is 115,659, the maximum RT (Response Time) is 605, and the average RT is 0.68.

sh producer.sh -t cluster-perf-tst16 -w 75 -s 1024 -n x.x.x.x:9876
Send TPS: 106813 Max RT: 605 Average RT:   0.687 Send Failed: 0 Response Failed: 0
Send TPS: 110828 Max RT: 605 Average RT:   0.673 Send Failed: 0 Response Failed: 1
Send TPS: 109855 Max RT: 605 Average RT:   0.676 Send Failed: 0 Response Failed: 3
Send TPS: 102741 Max RT: 605 Average RT:   0.730 Send Failed: 0 Response Failed: 3
Send TPS: 110123 Max RT: 605 Average RT:   0.681 Send Failed: 0 Response Failed: 3
Send TPS: 115659 Max RT: 605 Average RT:   0.648 Send Failed: 0 Response Failed: 3
Send TPS: 108157 Max RT: 605 Average RT:   0.693 Send Failed: 0 Response Failed: 3

Test Scenario 24 #

75 threads, message size of 3K, 16 queues. The maximum TPS for sending in the following results is 99,871, the maximum RT is 499, and the average RT is 0.78.

sh producer.sh -t cluster-perf-tst16 -w 75 -s 3072 -n x.x.x.x:9876
Send TPS: 90459 Max RT: 499 Average RT:   0.829 Send Failed: 0 Response Failed: 0
Send TPS: 96838 Max RT: 499 Average RT:   0.770 Send Failed: 0 Response Failed: 1
Send TPS: 96590 Max RT: 499 Average RT:   0.776 Send Failed: 0 Response Failed: 1
Send TPS: 95137 Max RT: 499 Average RT:   0.788 Send Failed: 0 Response Failed: 1
Send TPS: 89502 Max RT: 499 Average RT:   0.834 Send Failed: 0 Response Failed: 2
Send TPS: 90255 Max RT: 499 Average RT:   0.831 Send Failed: 0 Response Failed: 2
Send TPS: 99871 Max RT: 499 Average RT:   0.725 Send Failed: 0 Response Failed: 9

Test Scenario 25 #

100 threads, message size of 1K, 8 queues. The maximum TPS for sending in the following results is 126,590, the maximum RT is 402, and the average RT is 0.86.

sh producer.sh -t cluster-perf-tst8 -w 100 -s 1024 -n x.x.x.x:9876
Send TPS: 113204 Max RT: 402 Average RT:   0.883 Send Failed: 0 Response Failed: 0
Send TPS: 114872 Max RT: 402 Average RT:   0.868 Send Failed: 0 Response Failed: 1
Send TPS: 116261 Max RT: 402 Average RT:   0.860 Send Failed: 0 Response Failed: 1
Send TPS: 118116 Max RT: 402 Average RT:   0.847 Send Failed: 0 Response Failed: 1
Send TPS: 112594 Max RT: 402 Average RT:   0.888 Send Failed: 0 Response Failed: 1
Send TPS: 124407 Max RT: 402 Average RT:   0.801 Send Failed: 0 Response Failed: 2
Send TPS: 126590 Max RT: 402 Average RT:   0.790 Send Failed: 0 Response Failed: 2

Test Scenario 26 #

100 threads, message size of 3K, 8 queues. The maximum TPS for sending in the following results is 108,616, the maximum RT is 426, and the average RT is 0.93.

sh producer.sh -t cluster-perf-tst8 -w 100 -s 3072 -n x.x.x.x:9876
Send TPS: 106723 Max RT: 426 Average RT:   0.937 Send Failed: 0 Response Failed: 0
Send TPS: 104768 Max RT: 426 Average RT:   0.943 Send Failed: 0 Response Failed: 1
Send TPS: 106697 Max RT: 426 Average RT:   0.935 Send Failed: 0 Response Failed: 2
Send TPS: 105147 Max RT: 426 Average RT:   0.951 Send Failed: 0 Response Failed: 2
Send TPS: 105814 Max RT: 426 Average RT:   0.935 Send Failed: 0 Response Failed: 5
Send TPS: 108616 Max RT: 426 Average RT:   0.916 Send Failed: 0 Response Failed: 6
Send TPS: 101429 Max RT: 426 Average RT:   0.986 Send Failed: 0 Response Failed: 6

Test Scenario 27 #

100 threads, message size of 1K, 16 queues. The maximum TPS for sending in the following results is 123,424, the maximum RT is 438, and the average RT is 0.86.

sh producer.sh -t cluster-perf-tst16 -w 100 -s 1024 -n x.x.x.x:9876
Send TPS: 123424 Max RT: 438 Average RT:   0.805 Send Failed: 0 Response Failed: 0
Send TPS: 111418 Max RT: 438 Average RT:   0.897 Send Failed: 0 Response Failed: 0
Send TPS: 110360 Max RT: 438 Average RT:   0.905 Send Failed: 0 Response Failed: 0
Send TPS: 118734 Max RT: 438 Average RT:   0.842 Send Failed: 0 Response Failed: 0
Send TPS: 120725 Max RT: 438 Average RT:   0.816 Send Failed: 0 Response Failed: 4
Send TPS: 113823 Max RT: 438 Average RT:   0.878 Send Failed: 0 Response Failed: 4
Send TPS: 115639 Max RT: 438 Average RT:   0.865 Send Failed: 0 Response Failed: 4

Send TPS: 105010 Max RT: 535 Average RT:   1.488 Send Failed: 0 Response Failed: 2
Send TPS: 104541 Max RT: 535 Average RT:   1.496 Send Failed: 0 Response Failed: 2
Send TPS: 111285 Max RT: 535 Average RT:   1.425 Send Failed: 0 Response Failed: 3
Send TPS: 100228 Max RT: 535 Average RT:   1.585 Send Failed: 0 Response Failed: 4
Send TPS: 103343 Max RT: 535 Average RT:   1.536 Send Failed: 0 Response Failed: 5
Send TPS: 106607 Max RT: 535 Average RT:   1.473 Send Failed: 0 Response Failed: 5
Send TPS: 101537 Max RT: 535 Average RT:   1.561 Send Failed: 0 Response Failed: 6
Send TPS: 105061 Max RT: 535 Average RT: 1.428 Send Failed: 0 Response Failed: 0
Send TPS: 102117 Max RT: 535 Average RT: 1.465 Send Failed: 0 Response Failed: 1
Send TPS: 105569 Max RT: 535 Average RT: 1.421 Send Failed: 0 Response Failed: 1
Send TPS: 100689 Max RT: 535 Average RT: 1.489 Send Failed: 0 Response Failed: 2
Send TPS: 108464 Max RT: 535 Average RT: 1.381 Send Failed: 0 Response Failed: 2
Send TPS: 111285 Max RT: 535 Average RT: 1.348 Send Failed: 0 Response Failed: 2
Send TPS: 103406 Max RT: 535 Average RT: 1.451 Send Failed: 0 Response Failed: 2
Send TPS: 109203 Max RT: 535 Average RT: 1.388 Send Failed: 0 Response Failed: 2

Test Scenario 33 #

200 threads, message size of 1K, and 8 queues as the topic. The maximum TPS for the following results is 126,170, the maximum RT is 628, and the average RT is 1.71.

sh producer.sh -t cluster-perf-tst8 -w 200 -s 1024 -n x.x.x.x:9876
Send TPS: 117965 Max RT: 628 Average RT: 1.674 Send Failed: 0 Response Failed: 7
Send TPS: 115583 Max RT: 628 Average RT: 1.715 Send Failed: 0 Response Failed: 12
Send TPS: 118732 Max RT: 628 Average RT: 1.672 Send Failed: 0 Response Failed: 16
Send TPS: 126170 Max RT: 628 Average RT: 1.582 Send Failed: 0 Response Failed: 17
Send TPS: 116203 Max RT: 628 Average RT: 1.719 Send Failed: 0 Response Failed: 18
Send TPS: 114793 Max RT: 628 Average RT: 1.739 Send Failed: 0 Response Failed: 19

Test Scenario 34 #

200 threads, message size of 3K, and 8 queues as the topic. The maximum TPS for the following results is 110,892, the maximum RT is 761, and the average RT is 1.80.

sh producer.sh -t cluster-perf-tst8 -w 200 -s 3072 -n x.x.x.x:9876
Send TPS: 107240 Max RT: 761 Average RT: 1.865 Send Failed: 0 Response Failed: 0
Send TPS: 104585 Max RT: 761 Average RT: 1.906 Send Failed: 0 Response Failed: 2
Send TPS: 110892 Max RT: 761 Average RT: 1.803 Send Failed: 0 Response Failed: 2
Send TPS: 105414 Max RT: 761 Average RT: 1.898 Send Failed: 0 Response Failed: 2
Send TPS: 105904 Max RT: 761 Average RT: 1.885 Send Failed: 0 Response Failed: 3
Send TPS: 110748 Max RT: 761 Average RT: 1.806 Send Failed: 0 Response Failed: 3

Test Scenario 35 #

200 threads, message size of 1K, and 16 queues as the topic. The maximum TPS for the following results is 124,760, the maximum RT is 601, and the average RT is 1.63.

sh producer.sh -t cluster-perf-tst16 -w 200 -s 1024 -n x.x.x.x:9876
Send TPS: 118892 Max RT: 601 Average RT: 1.679 Send Failed: 0 Response Failed: 4
Send TPS: 118839 Max RT: 601 Average RT: 1.668 Send Failed: 0 Response Failed: 12
Send TPS: 117122 Max RT: 601 Average RT: 1.704 Send Failed: 0 Response Failed: 12
Send TPS: 122670 Max RT: 601 Average RT: 1.630 Send Failed: 0 Response Failed: 12
Send TPS: 119592 Max RT: 601 Average RT: 1.672 Send Failed: 0 Response Failed: 12
Send TPS: 121243 Max RT: 601 Average RT: 1.649 Send Failed: 0 Response Failed: 12
Send TPS: 124760 Max RT: 601 Average RT: 1.603 Send Failed: 0 Response Failed: 12
Send TPS: 124354 Max RT: 601 Average RT: 1.608 Send Failed: 0 Response Failed: 12
Send TPS: 119272 Max RT: 601 Average RT: 1.677 Send Failed: 0 Response Failed: 12

Test Scenario 36 #

200 threads, message size of 3K, and 16 queues as the topic. The maximum TPS for the following results is 111,201, the maximum RT is 963, and the average RT is 1.88.

sh producer.sh -t cluster-perf-tst16 -w 200 -s 3072 -n x.x.x.x:9876
Send TPS: 105091 Max RT: 963 Average RT: 1.896 Send Failed: 0 Response Failed: 4
Send TPS: 106243 Max RT: 963 Average RT: 1.882 Send Failed: 0 Response Failed: 4
Send TPS: 103994 Max RT: 963 Average RT: 1.958 Send Failed: 0 Response Failed: 5
Send TPS: 109741 Max RT: 963 Average RT: 1.822 Send Failed: 0 Response Failed: 5
Send TPS: 103788 Max RT: 963 Average RT: 1.927 Send Failed: 0 Response Failed: 5
Send TPS: 110597 Max RT: 963 Average RT: 1.805 Send Failed: 0 Response Failed: 6
Send TPS: 111201 Max RT: 963 Average RT: 1.798 Send Failed: 0 Response Failed: 6

Summary #

Based on the performance tests above, the highest TPS achieved was 126,170. We can conclude that the theoretical capacity of the cluster is around 120,000. Daily traffic should be controlled at around 40,000. When it exceeds 40,000, new topics should be allocated to other clusters.