The JMS specification supports two types of message delivery: persistent and non-persistent. A message delivered with the persistent delivery property must be logged to stable storage. For nonpersistent messages, a JMS provider must make best efforts to deliver the message, but it won’t be logged to stable storage.
ActiveMQ supports both of these types of message delivery and can also be configured to support message recovery, an in-between state where messages are cached in memory. ActiveMQ supports a pluggable strategy for message storage and provides storage options for in-memory, file-based, and relational databases.
Persistent messages are used if you want messages to always be available to a message consumer after they’ve been delivered to the broker, even if that consumer isn’t running when the message was sent. Once a message has been consumed and acknowledged by a message consumer, it’s typically deleted from the broker’s message store.
Nonpersistent messages are typically used for sending notifications or real-time data. You should use nonpersistent messages when performance is critical and guaranteed delivery of the message isn’t required.
How are messages stored by ActiveMQ?
It’s important to gain some basic knowledge of the storage mechanisms for messages in an ActiveMQ message store. This will aid in configuration and provide an awareness of what takes place in the ActiveMQ broker during the delivery of persistent messages. Messages sent to queues and topics are stored differently, because there are some storage optimizations that can be made with topics that don’t make sense with queues, as we’ll explain.
Storage for queues is straightforward—messages are basically stored in first in, first out order (FIFO). See figure 5.1 for a depiction of this. One message is dispatched to a single consumer at a time. Only when that message has been consumed and acknowledged can it be deleted from the broker’s message store.
For durable subscribers to a topic, each consumer gets a copy of the message. In order to save storage space, only one copy of a message is stored by the broker. A durable subscriber object in the store maintains a pointer to its next stored message and dispatches a copy of it to its consumer as shown in figure 5.2. The message store is implemented in this manner because each durable subscriber could be consuming messages at different rates or they may not all be running at the same time. Also, because every message can potentially have many consumers, a message can’t be deleted from the store until it’s been successfully delivered to every interested durable subscriber.
The KahaDB message store:
The recommended message store for general-purpose messages since ActiveMQ version 5.3 is KahaDB. This is a file-based message store that combines a transactional journal, for reliable message storage and recovery, with good performance and scalability.
The KahaDB store is a file-based, transactional store that’s been tuned and designed for the fast storage of messages. The aim of the KahaDB store is to be easy to use and as fast as possible. Its use of a file-based message database means there’s no prerequisite for a third-party database. This message store enables ActiveMQ to be downloaded and running in literally minutes. In addition, the structure of the KahaDB store has been streamlined especially for the requirements of a message broker.
The KahaDB message store uses a transactional log for its indexes and only uses one index file for all its destinations. It’s been used in production environments with 10,000 active connections, each connection having a separate queue. The configurability of the KahaDB store means that it can be tuned for most usage scenarios, from high throughput applications (for example, trading platforms), to storing large amounts of messages (for example, GPS tracking).
To enable the KahaDB store for ActiveMQ, you need to configure the
If you want to embed an ActiveMQ broker inside an application, the message store can also be configured programmatically. Here’s an example of a programmatic configuration for KahaDB:
Although the example seems small, it’s enough to create an ActiveMQ broker using the KahaDB message store and listen for ActiveMQ clients connecting over TCP. For more information about embedding ActiveMQ, see chapter 8.
The KahaDB message store internals
The KahaDB message store is the fastest of all the provided message store implementations. Its speed is the result of the combination of a fast transactional journal comprised of data log files, the highly optimized indexing of message IDs, and in-memory message caching. Figure 5.3 provides a high-level diagram of the KahaDB message store.
The diagram provides a view of the three distinct parts of the KahaDB message store including the following:
- The data logs
- The cache
- The BTree indexes
The KahaDB message store directory structure:
When you start an ActiveMQ broker configured to use a KahaDB store, a directory will automatically be created in which the persistent messages are stored. This directory structure is shown in figure 5.4.
Inside of the KahaDB directory, the following directory and file structures can be found:
- db log files
- archive directory
Configuring the KahaDB message store:
The KahaDB message store can be configured in the activemq.xml file. Its configuration options control the different tuning parameters (table 5.1):
Table 5.1 Configuration options available for the KahaDB message store
ActiveMQ provides a pluggable API for message stores, and there are three additional implementations to KahaDB that are shipped with ActiveMQ:
We’ll look at the use cases and configuration for these additional message stores in the next three sections.