Convert Figma logo to code with AI

apache logorocketmq

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

21,052
11,614
21,052
427
View on GitHub

Top Related Projects

apache's avatar

kafka

28,601

Mirror of Apache Kafka

apache's avatar

pulsar

14,185

Apache Pulsar - distributed pub-sub messaging system

rabbitmq's avatar

rabbitmq-server

12,193

Open source RabbitMQ: core server and tier 1 (built-in) plugins

confluentinc's avatar

confluent-kafka-go

4,582

Confluent's Apache Kafka Golang client

apache's avatar

activemq

2,303

Mirror of Apache ActiveMQ

Quick Overview

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance, and reliability, scalability, and flexibility. It supports various messaging patterns and can handle trillions of messages per day, making it suitable for large-scale, high-volume scenarios.

Pros

  • High throughput and low latency, capable of processing millions of messages per second
  • Supports multiple messaging patterns (pub/sub, P2P, broadcasting, etc.)
  • Highly scalable and fault-tolerant architecture
  • Rich features like message tracing, transaction messages, and scheduled messages

Cons

  • Steeper learning curve compared to some simpler messaging systems
  • Configuration and setup can be complex for beginners
  • Limited ecosystem compared to some more established messaging platforms
  • Documentation can be inconsistent or outdated in some areas

Code Examples

  1. Sending a message:
DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
producer.setNamesrvAddr("localhost:9876");
producer.start();

Message msg = new Message("TopicTest", "TagA", "Hello RocketMQ".getBytes(RemotingHelper.DEFAULT_CHARSET));
SendResult sendResult = producer.send(msg);

System.out.printf("%s%n", sendResult);
producer.shutdown();
  1. Consuming messages:
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("ConsumerGroupName");
consumer.setNamesrvAddr("localhost:9876");
consumer.subscribe("TopicTest", "*");
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
        System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();
  1. Sending a transaction message:
TransactionListener transactionListener = new TransactionListenerImpl();
TransactionMQProducer producer = new TransactionMQProducer("TransactionProducerGroup");
producer.setNamesrvAddr("localhost:9876");
producer.setTransactionListener(transactionListener);
producer.start();

Message msg = new Message("TopicTest", "TagA", "Hello RocketMQ Transaction".getBytes(RemotingHelper.DEFAULT_CHARSET));
SendResult sendResult = producer.sendMessageInTransaction(msg, null);

Getting Started

  1. Download and install RocketMQ from the official website.
  2. Start the Name Server: 
    nohup sh bin/mqnamesrv &
  3. Start the Broker: 
    nohup sh bin/mqbroker -n localhost:9876 &
  4. Add RocketMQ dependency to your project (Maven example): 
    <dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-client</artifactId>
    <version>4.9.3</version>
</dependency>
  5. Use the code examples above to start sending and consuming messages.

Competitor Comparisons

apache logo

kafka

28,601

Mirror of Apache Kafka

Pros of Kafka

  • Higher throughput and lower latency for large-scale data streaming
  • More mature ecosystem with extensive tooling and integrations
  • Better support for stream processing and real-time analytics

Cons of Kafka

  • More complex setup and configuration
  • Higher resource consumption, especially for smaller deployments
  • Steeper learning curve for developers and operators

Code Comparison

Kafka producer example:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
Producer<String, String> producer = new KafkaProducer<>(props);

RocketMQ producer example:

DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
producer.setNamesrvAddr("localhost:9876");
producer.start();
Message msg = new Message("TopicTest", "TagA", "Hello RocketMQ".getBytes(RemotingHelper.DEFAULT_CHARSET));
SendResult sendResult = producer.send(msg);

Both Kafka and RocketMQ are powerful distributed messaging systems, but they have different strengths and use cases. Kafka excels in high-throughput scenarios and stream processing, while RocketMQ offers simpler setup and lower resource requirements for smaller deployments. The choice between them depends on specific project needs and scalability requirements.

apache logo

pulsar

14,185

Apache Pulsar - distributed pub-sub messaging system

Pros of Pulsar

  • Multi-tenancy support with better isolation between tenants
  • Built-in support for geo-replication and disaster recovery
  • More flexible storage options, including tiered storage

Cons of Pulsar

  • Higher complexity and steeper learning curve
  • Potentially higher resource requirements for smaller deployments

Code Comparison

RocketMQ producer example:

DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
producer.start();
Message msg = new Message("TopicTest", "TagA", "Hello RocketMQ".getBytes(RemotingHelper.DEFAULT_CHARSET));
SendResult sendResult = producer.send(msg);

Pulsar producer example:

PulsarClient client = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Producer<byte[]> producer = client.newProducer().topic("my-topic").create();
producer.send("Hello Pulsar".getBytes());

Both RocketMQ and Pulsar are robust messaging systems, but they have different strengths. RocketMQ is often simpler to set up and use, making it a good choice for smaller deployments or teams new to message queues. Pulsar offers more advanced features and scalability options, making it suitable for larger, more complex distributed systems.

rabbitmq logo

rabbitmq-server

12,193

Open source RabbitMQ: core server and tier 1 (built-in) plugins

Pros of RabbitMQ

  • More mature and widely adopted, with extensive documentation and community support
  • Supports multiple messaging protocols (AMQP, MQTT, STOMP)
  • Offers a user-friendly management interface for monitoring and administration

Cons of RabbitMQ

  • Generally lower throughput compared to RocketMQ, especially for large-scale scenarios
  • Less focus on distributed systems and big data processing capabilities
  • More complex setup and configuration for high availability and clustering

Code Comparison

RabbitMQ (Erlang):

basic_publish(Channel, <<"my_exchange">>, <<"routing_key">>, <<"Hello, World!">>),
{#'basic.get_ok'{}, Content} = basic_get(Channel, <<"my_queue">>, no_ack),
io:format("Received: ~p~n", [Content]).

RocketMQ (Java):

producer.send(new Message("TopicTest", "Hello, World!".getBytes()));
MessageExt msg = consumer.receive(1000);
System.out.printf("Received: %s%n", new String(msg.getBody()));

Both RabbitMQ and RocketMQ are powerful message queue systems, but they have different strengths. RabbitMQ excels in flexibility and ease of use, while RocketMQ focuses on high performance and scalability for large-scale distributed systems. The choice between them depends on specific project requirements and use cases.

confluentinc logo

confluent-kafka-go

4,582

Confluent's Apache Kafka Golang client

Pros of confluent-kafka-go

  • Native Go implementation, offering better performance and integration with Go ecosystems
  • Supports both producer and consumer APIs, providing a comprehensive solution for Kafka interactions
  • Actively maintained by Confluent, ensuring up-to-date features and compatibility with latest Kafka versions

Cons of confluent-kafka-go

  • Limited to Kafka messaging, while RocketMQ offers a broader range of messaging patterns
  • Steeper learning curve for developers not familiar with Kafka concepts
  • Less flexibility in terms of message storage and retrieval compared to RocketMQ's more diverse options

Code Comparison

confluent-kafka-go:

producer, err := kafka.NewProducer(&kafka.ConfigMap{"bootstrap.servers": "localhost"})
producer.Produce(&kafka.Message{
    TopicPartition: kafka.TopicPartition{Topic: &topic, Partition: kafka.PartitionAny},
    Value:          []byte(value),
}, nil)

RocketMQ:

DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
producer.setNamesrvAddr("localhost:9876");
producer.start();
Message msg = new Message("TopicTest", "TagA", "OrderID188", "Hello world".getBytes(RemotingHelper.DEFAULT_CHARSET));
SendResult sendResult = producer.send(msg);

Both examples demonstrate basic message production, but RocketMQ's API is more Java-centric, while confluent-kafka-go provides a more Go-idiomatic approach.

apache logo

activemq

2,303

Mirror of Apache ActiveMQ

Pros of ActiveMQ

  • More mature and established project with a longer history
  • Supports multiple protocols (AMQP, MQTT, STOMP) out of the box
  • Offers a wider range of deployment options, including standalone broker and in-memory broker

Cons of ActiveMQ

  • Generally lower throughput and higher latency compared to RocketMQ
  • More complex configuration and setup process
  • Less scalable for high-volume messaging scenarios

Code Comparison

ActiveMQ (Java):

ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61616");
Connection connection = connectionFactory.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue("TEST.FOO");
MessageProducer producer = session.createProducer(destination);

RocketMQ (Java):

DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
producer.setNamesrvAddr("localhost:9876");
producer.start();
Message msg = new Message("TopicTest", "TagA", "Hello RocketMQ".getBytes(RemotingHelper.DEFAULT_CHARSET));
SendResult sendResult = producer.send(msg);

Both ActiveMQ and RocketMQ are popular message queue solutions, but RocketMQ generally offers better performance and scalability for high-volume scenarios. ActiveMQ provides more flexibility in terms of protocols and deployment options, making it suitable for a wider range of use cases.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Apache RocketMQ

Build Status CodeCov Maven Central Release License Average Time to Resolve An Issue Percentage of Issues Still Open Twitter Follow

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability.

It offers a variety of features:

  • Messaging patterns including publish/subscribe, request/reply and streaming
  • Financial grade transactional message
  • Built-in fault tolerance and high availability configuration options base on DLedger Controller
  • Built-in message tracing capability, also support opentracing
  • Versatile big-data and streaming ecosystem integration
  • Message retroactivity by time or offset
  • Reliable FIFO and strict ordered messaging in the same queue
  • Efficient pull and push consumption model
  • Million-level message accumulation capacity in a single queue
  • Multiple messaging protocols like gRPC, MQTT, JMS and OpenMessaging
  • Flexible distributed scale-out deployment architecture
  • Lightning-fast batch message exchange system
  • Various message filter mechanics such as SQL and Tag
  • Docker images for isolated testing and cloud isolated clusters
  • Feature-rich administrative dashboard for configuration, metrics and monitoring
  • Authentication and authorization
  • Free open source connectors, for both sources and sinks
  • Lightweight real-time computing

Quick Start

This paragraph guides you through steps of installing RocketMQ in different ways. For local development and testing, only one instance will be created for each component.

Run RocketMQ locally

RocketMQ runs on all major operating systems and requires only a Java JDK version 8 or higher to be installed. To check, run java -version:

$ java -version
java version "1.8.0_121"

For Windows users, click here to download the 5.2.0 RocketMQ binary release, unpack it to your local disk, such as D:\rocketmq. For macOS and Linux users, execute following commands:

# Download release from the Apache mirror
$ wget https://dist.apache.org/repos/dist/release/rocketmq/5.2.0/rocketmq-all-5.2.0-bin-release.zip

# Unpack the release
$ unzip rocketmq-all-5.2.0-bin-release.zip

Prepare a terminal and change to the extracted bin directory:

$ cd rocketmq-all-5.2.0-bin-release/bin

1) Start NameServer

NameServer will be listening at 0.0.0.0:9876, make sure that the port is not used by others on the local machine, and then do as follows.

For macOS and Linux users:

### start Name Server
$ nohup sh mqnamesrv &

### check whether Name Server is successfully started
$ tail -f ~/logs/rocketmqlogs/namesrv.log
The Name Server boot success...

For Windows users, you need set environment variables first:

  • From the desktop, right click the Computer icon.
  • Choose Properties from the context menu.
  • Click the Advanced system settings link.
  • Click Environment Variables.
  • Add Environment ROCKETMQ_HOME="D:\rocketmq".

Then change directory to rocketmq, type and run:

$ mqnamesrv.cmd
The Name Server boot success...

2) Start Broker

For macOS and Linux users:

### start Broker
$ nohup sh bin/mqbroker -n localhost:9876 &

### check whether Broker is successfully started, eg: Broker's IP is 192.168.1.2, Broker's name is broker-a
$ tail -f ~/logs/rocketmqlogs/broker.log
The broker[broker-a, 192.169.1.2:10911] boot success...

For Windows users:

$ mqbroker.cmd -n localhost:9876
The broker[broker-a, 192.169.1.2:10911] boot success...

Run RocketMQ in Docker

You can run RocketMQ on your own machine within Docker containers, host network will be used to expose listening port in the container.

1) Start NameServer

$ docker run -it --net=host apache/rocketmq ./mqnamesrv

2) Start Broker

$ docker run -it --net=host --mount source=/tmp/store,target=/home/rocketmq/store apache/rocketmq ./mqbroker -n localhost:9876

Run RocketMQ in Kubernetes

You can also run a RocketMQ cluster within a Kubernetes cluster using RocketMQ Operator. Before your operations, make sure that kubectl and related kubeconfig file installed on your machine.

1) Install CRDs

### install CRDs
$ git clone https://github.com/apache/rocketmq-operator
$ cd rocketmq-operator && make deploy

### check whether CRDs is successfully installed
$ kubectl get crd | grep rocketmq.apache.org
brokers.rocketmq.apache.org                 2022-05-12T09:23:18Z
consoles.rocketmq.apache.org                2022-05-12T09:23:19Z
nameservices.rocketmq.apache.org            2022-05-12T09:23:18Z
topictransfers.rocketmq.apache.org          2022-05-12T09:23:19Z

### check whether operator is running
$ kubectl get pods | grep rocketmq-operator
rocketmq-operator-6f65c77c49-8hwmj   1/1     Running   0          93s

2) Create Cluster Instance

### create RocketMQ cluster resource
$ cd example && kubectl create -f rocketmq_v1alpha1_rocketmq_cluster.yaml

### check whether cluster resources is running
$ kubectl get sts
NAME                 READY   AGE
broker-0-master      1/1     107m
broker-0-replica-1   1/1     107m
name-service         1/1     107m

Apache RocketMQ Community

Learn it & Contact us

Contributing

We always welcome new contributions, whether for trivial cleanups, big new features or other material rewards, more details see here.

License

Apache License, Version 2.0 Copyright (C) Apache Software Foundation

Export Control Notice

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software:

This software uses Apache Commons Crypto (https://commons.apache.org/proper/commons-crypto/) to support authentication, and encryption and decryption of data sent across the network between services.