• notice
  • Congratulations on the launch of the Sought Tech site

How to install and deploy a standalone version of Rocket with Docker

In this article, the editor will introduce "how to install and deploy the stand-alone version of Rocket in Docker" in detail. The content is detailed, the steps are clear, and the details are handled properly. I hope this article "How to install and deploy the stand-alone version of Rocket in Docker" can help you solve your doubts. Follow the editor below. The idea of is slowly deepened, and let's learn new knowledge together.

foreword

At present, the mainstream MQs are mainly Rocketmq, kafka, and Rabbitmq. Compared with Rabbitmq and kafka, Rocketmq has the following main advantages:

  1. Support for transactional messages (capable of solving distributed transaction problems)

  2. Support sequential messages (the bottom layer has been implemented using memory queues)

  3. Support consumer-side tag filtering to reduce unnecessary network transmission

Rocketmq is an upgraded version of the kafka implementation.

Advantages and disadvantages

Advantages of RocketMQ

  • Single machine throughput: 100,000 class

  • Availability: very high, distributed architecture

  • Message reliability: After parameter optimization configuration, messages can be lost 0

  • Functional support: MQ has relatively complete functions, is still distributed, and has good scalability

  • Supports 1 billion-level message stacking without performance degradation due to stacking

  • The source code is Java, which is convenient for secondary development combined with the company's own business.

  • Born for the financial Internet field, for scenarios with high reliability requirements, especially order deductions in e-commerce, and business peak shaving, when a large number of transactions influx, the backend may not be able to handle the situation in time

  • RoketMQ may be more reliable in terms of stability. These business scenarios have undergone many tests on Alibaba's Double 11.

Disadvantages of RocketMQ

  • Not many client languages are supported, currently only Java and C++ are supported, and C++ is immature

  • Interfaces such as JMS are not implemented in the MQ core, and some systems need to modify a lot of code to migrate

Introduction to common nouns

  • provider

The message producer is located in the user's process. The producer obtains the routing information of all brokers through the NameServer, selects which broker to send the message to according to the load balancing strategy, and then calls the broker interface to submit the message.

  • provider group

A producer group, in simple terms, is a producer group that multiple producers that send the same type of message.

  • consumer

Message consumer, located in the user process. After the Consumer obtains the routing information of all brokers through the NameServer, it sends a Pull request to the Broker to obtain the message data. Consumers can be started in two modes, Broadcast and Cluster. In broadcast mode, a message will be sent to all consumers, while in cluster mode, a message will only be sent to one consumer.

  • consumer group

A consumer group, similar to a producer, consists of multiple Consumer instances that consume the same type of messages to form a consumer group.

  • Topic Topic

It is used to divide messages by topic. The Producer sends the message to the specified Topic, and the Consumer can receive the message by subscribing to the Topic. Topic has no strong relationship with the sender and the consumer. The sender can send messages to multiple topics at the same time, and the consumer can also subscribe to the messages of multiple topics. In RocketMQ, Topic is a logical concept. Message storage is not separated by topic.

  • Message

Represents a message and is uniquely identified by MessageId. Users can set messageKey when sending, which is convenient for later query and tracking. A Message must specify Topic, which is equivalent to the mailing address. Message also has an optional Tag setting so that the consumer can filter messages based on Tag. Additional key-value pairs can also be added, for example you need a business key to look up messages on the Broker, which is convenient for diagnosing problems during development.

  • Tag

It can be considered as a further refinement of Topic. Generally, messages of different purposes are marked by introducing tags in the same business module.

  • Broker Broker

It is the core module of RocketMQ, responsible for receiving and storing messages, and providing Push/Pull interfaces to send messages to Consumer. Consumer can choose to read data from Master or Slave. Multiple masters/slaves form a Broker cluster, and there is no data interaction between the Master nodes in the cluster. Broker also provides the function of message query, which can query messages through MessageID and MessageKey. Borker will synchronize its own Topic configuration information to NameServer in real time.

  • Queue Topic

Queue Topic and Queue have a one-to-many relationship. A Topic can contain multiple Queues, which are mainly used for load balancing. When sending a message, the user only specifies the Topic, and the Producer will choose which Queue to send to according to the routing information of the Topic. When a Consumer subscribes to a message, it will decide which Queue messages to subscribe to according to the load balancing strategy.

  • Offset

When RocketMQ stores messages, it will generate a message index file for each Queue under each topic, and each Queue corresponds to an Offset to record the number of messages in the current Queue.

  • NameServer

NameServer can be regarded as the registration center of RocketMQ, which manages two parts of data: the routing configuration of the Topic-Queue of the cluster; the real-time configuration information of the Broker. Other modules obtain the latest topic configuration and routing information through the interface provided by the Nameserver.

  • Producer/Consumer

Obtain the address information of the Broker corresponding to the topic through the query interface

  • Broker

Register configuration information to NameServer, update topic information to NameServer in real time

Installation and deployment of stand-alone version

This is used for learning, so use docker to install, and the installation speed is fast.

1. Install and start the nameserverservice

 docker run -d -p  9876 : 9876 -v /opt/ rocketmq  /data/namesrv/logs:/root/logs -v /opt/rocketmq/data/namesrv/store:/root/store --name rmqnamesrv -e  " MAX_POSSIBLE_HEAP=100000000"  rocketmqinc/rocketmq sh mqnamesrv

2. Install and start the broker service

According to the container volume configuration, add the broker.conf content in the specified location /opt/rocketmq/conf/broker.conf . (may not be needed)

brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
brokerIP1 = docker host IP

Start the broker service

docker  run -d -p  10911 : 10911  -p  10909 : 10909  -v /opt/rocketmq/data/broker/logs:/root/logs -v /opt/rocketmq/rocketmq/data/broker/store:/root/store -v /opt/rocketmq/conf/broker.conf:/opt/rocketmq/conf/broker.conf --name rmqbroker --link rmqnamesrv:namesrv -e  "NAMESRV_ADDR=namesrv:9876"  -e  "MAX_POSSIBLE_HEAP=200000000"  rocketmqinc /rocketmq sh mqbroker -c /opt/rocketmq/conf/broker.conf

3. Install the startup console

docker run -d -e  "JAVA_OPTS=-Drocketmq.config.namesrvAddr=nameserIP:9876 -Drocketmq.config.isVIPChannel=false"  -p  8080 : 8080  --name rmqconsole -t styletang/rocketmq- console -ng

Cluster deployment

Cluster Deployment Schematic

62dfb7c89ca85.png

Broker master and backup

host:

Standby: Synchronize the data of the master node. When the master node goes down, it can be changed to the master to ensure high availability.

In rocketmq, it is divided into multiple masters and multiple slaves to achieve fragmented storage of our topic data.

Four cluster deployment methods

  • The disadvantage of a single Master node is that the entire service may be unavailable if it goes down;

  • Multiple Master nodes share and store our messages. Disadvantages: There is no Slave node, and the message data may be lost after the main Master node goes down;

  • Multiple Master and Slave nodes are very efficient in asynchronous form, and data may cause short delays (millisecond level)

  • Multiple Master and Slave nodes use synchronization, which has low efficiency and no data delay.

Cluster deployment

The deployment of the cluster mainly depends on the configuration file. After the nameserver is deployed, the connection information of the same nameserver cluster is configured in the configuration files of multiple brokers, and the broker cluster is established. Then Rocketmq-console can connect to one of the nameservers.

Configuration file description:

#Cluster name, you can distinguish different clusters, different businesses can build multiple clusters, the value in the configuration files of multiple brokers must be the same
brokerClusterName=kaico

# Broker's name, Master and Slave indicate the relationship by using the same Broker name to indicate which Master's Slave a Slave is.
brokerName=broker-a
# A Master Barker can have multiple Slaves, 0 means Master, greater than 0 means different Slave IDs.
brokerId=0 

# Echoes with the fileReservedTim parameter, indicating at what time to delete the message, the default value of 04 means 4:00 in the morning.
deleteWhen=04
namesrvAddr=mqnameserver1:9876;mqnameserver2:9876
autoCreateTopicEnable=true #Whether to automatically create a topic (true is to allow automatic cluster creation)
#topic default number of queues created
defaultTopicQueueNums=4 #Whether
to allow Broker to automatically create subscription groups, it is recommended to go offline Open, close online, default [true]
autoCreateSubscriptionGroup=true
#Broker listens on the port number. If multiple Brokers are started on a machine, different port numbers should be set to avoid conflicts.
listenPort=10911
brokerIp=192.168.1.1

1. First start two nameserver services

2. Start multiple broker services (two are arranged here)

The first broker configuration file

brokerClusterName=kaico #Cluster name
brokerName=broker-a #broker name
brokerId=0 #identification of master node (0 means master node)
namesrvAddr=mqnameserver1:9876;mqnameserver2:9876 #nameServer cluster

Second broker configuration file

brokerClusterName=kaico #Cluster name
brokerName=broker-b #broker name
brokerId=0 #identification of master node (0 means master node)
namesrvAddr=mqnameserver1:9876;mqnameserver2:9876 #nameServer cluster

Start method:

  1. Start the nameServer service first

  2. Then start the broker, and you can specify the connection information of the nameServer in the command to start the broker.

After reading this, the article "How to Install and Deploy Standalone Rocket in Docker" has been introduced. If you want to master the knowledge points of this article, you need to practice and use it yourself to understand.


Tags

Technical otaku

Sought technology together

Related Topic

0 Comments

Leave a Reply

+