Overview
In many systems, having multiple consumers helps process tasks faster, especially tasks that take a long time to complete.
For example, your job is to convert hundreds of thousands docx files to pdf format, you may want to have many consumers instead of one.
In this post, I’m going to show you
- How you can register multiple consumers to a topic
- How to make sure messages only consumed by ONE consumer only
Let’s get started.
Create multiple consumers
It is quite simple to create multiple consumers for a queue. Simply annotate any method with @JmsListener
@JmsListener(destination = "package_queue", id = "package_consumer_1")
public void packageReceiver1(String packageData) {
System.out.printf("In receiver 1, receiving package %s%n", packageData);
}
@JmsListener(destination = "package_queue", id = "package_consumer_2")
public void packageReceiver2(String packageData) {
System.out.printf("In receiver 2, receiving package %s%n", packageData);
}
In the code example above, I created two consumers. The key point here is to have both of them register to the same destination (queue).
Let’s create a producer and send multiple messages to this queue:
@Autowired
JmsTemplate template;
@GetMapping("/send-package-simple")
public String sendPackageSimple() {
for (int i = 0; i < 10; i++) {
template.convertAndSend("package_queue", "Package #" + i);
}
return "packages sent!";
}
As in previous tutorials of this series, I create an endpoint so I can query to send the messages. In the code example above, I sent 10 messages to the queue named package_queue.
Let’s run the producer and see how the consumers work:
As you can see, I was able to make the request.
Let’s check the consumers’ log to see how the consumers handle the messages:
As you can see, the messages were consumed evenly by the two consumers.
However, this is not the ideal scenario in some cases. Imagine you need to send a big package by splitting it into 10 pieces. You want all the pieces to be processed by one consumer but you don’t care which one. Because if the pieces of this package are distributed between consumers, it is not possible to assemble them back to the original package.
In real life, you encounter this issue when sending files, for example. For big files, you want to split them into small pieces before sending them. On the receiving end, you need all the pieces to be available in one place to assemble them back.
There is a solution for that, which I’ll show you in the next section.
Using JMS Message Group
To quote the official documentation:
The Message Group feature then ensures that all messages for the same message group will be sent to the same JMS consumer – whilst that consumer stays alive. As soon as the consumer dies another will be chosen.
https://activemq.apache.org/message-groups
That means if you want a group of messages to be consumed by only one consumer, you need to configure them to have the same group id.
With ActiveMQ, that can be done by using the property JMSXGroupID.
You configure this property on the producer, not on the consumer.
So, to make sure ten packages are consumed by one consumer, you need to rewrite the producer method as follows:
@GetMapping("/send-package-group")
public String sendPackageGroup() {
var gid = UUID.randomUUID().toString();
for (int i = 0; i < 10; i++) {
int finalI = i;
template.send("package_queue", session -> {
var m = session.createTextMessage("Package " + finalI);
m.setStringProperty("JMSXGroupID", gid);
return m;
});
}
return "packages sent!";
}
As you can see, on line 3, I generated a random UUID to use as group id. This ensure different requests will have different group id. Why? if you set the group Id to a fixed value, ALL messages will be consumed by a single consumer. That defeats the use of multiple consumers.
On line 6, you can see that I didn’t use convertAndSend as previously. Instead, I used the send method. This allows me to set the group id property on line 8.
That is all you need to do to achieve the desired result.
Let’s restart the producer and consumer and try making some GET requests.
This is the log on the consumer side:
As you can see, consumer 1 receives all the packages of a request and so does consumer 2. However, for a single GET request, all messages are consumed by ONE consumer.
Conclusion
In this post, I’ve shown you how to configure the JMSXGroupID property so all messages with this property are consumed by one consumer.
The source code of this post is available on GitHub. Specifically:
I build softwares that solve problems. I also love writing/documenting things I learn/want to learn.