[ InAction Note ] Ch5. Message Storage - The AMQ message store

Preface: 
The AMQ message store, like KahaDB, is a combination of a transactional journal for reliable persistence (to survive system crashes) and high-performance indexes, which makes this store the best option when message throughput is the main requirement for an application. But because it uses two separate files for every index, and there’s an index per destination, the AMQ message store shouldn’t be used if you intend to use thousands of queues per broker. Also, recovery can be slow if the ActiveMQ broker isn’t shut down cleanly. 

The AMQ message store internals: 
The main components of the AMQ message store are similar to that of the KahaDB message store, in that there’s a cache, message data logs, and a reference store for accessing the data logs in order. Figure 5.5 provides a high-level diagram of the AMQ message store. 
 

The diagram provides a view of the three distinct parts of the AMQ message store: 
- The data logs: These act as a message journal. 
- The cache: This holds messages for fast retrieval in memory after they’ve been written to the data logs. 
- The reference store: This holds references to the messages in the journal that are indexed by their message ID. 

The AMQ message store directory structure: 
When you start ActiveMQ with the AMQ message store configured, a directory will automatically be created in which the persistent messages are held. The AMQ message store directory contains subdirectories for all the brokers that are running on the machine. For this reason it’s strongly recommended that each broker use a unique name. In the default configuration for ActiveMQ, the broker name is localhost, which needs to changed to something unique. This directory structure is represented in figure 5.6—the AMQ store directory structure. 

The following directories and files can be found inside the data directory of an ActiveMQ broker: 
- A lock file 

Ensures that only one broker can access this data at any given time. The lock is commonly used for hot standby purposes where more than one bro-
ker with the same name will exist on the same system.

- A temp-storage directory 

Used for storing nonpersistent messages that can no longer be stored in broker memory. These messages are typically awaiting delivery to a slow consumer.

- The kr-store 

The directory structure used by the reference (index) part of the AMQ message store. It uses the Kaha reference store by default (Kaha is part of the ActiveMQ core library) to index and store references to messages in the data logs. There are two distinct parts to the kr-store:
* The data directory

Contains the indexes and collections used to reference the messages held in the data logs. This data directory is deleted and rebuilt as part of recovery, if the broker hasn’t shut down cleanly. You can force recovery by manually deleting this directory before starting the broker.

* The state directory

Holds information about durable topic consumers. The journal itself doesn’t hold information about consumers, so when it’s recovered it has to retrieve information about the durable subscribers first to accurately rebuild its database.

- The journal directory 

Contains the data files for the data logs, and a data-control file that holds some meta information. The data files are reference counted, so when all the contained messages are delivered, a data file can be deleted or archived.

- The archive directory 

Exists only if archiving is enabled. Its default location can be found next to the journal. It makes sense to use a separate partition or disk. The archive is used to store data logs from the journal directory, which are moved here instead of being deleted. This makes it possible to replay messages from the archive at a later point. To replay messages, move the archived data logs (or a subset) to a new journal directory and start a new broker pointed to the location of this directory. It’ll automatically replay the data logs in the journal.

Configuring the AMQ message store: 
The AMQ store configuration allows the user to change its basic behaviour around indexing, checkpoint intervals, and the size of the journal data files. These items and many more can be customized through the use of properties. The key properties for the AMQ store are shown in table 5.2. 

Table 5.2 Configuration properties for the AMQ message store 

Below is an example of using the properties from table 5.2 in an ActiveMQ XML configuration file: 

This is but a small example of a customized configuration for the AMQ store using the available properties.