程式扎記: [ InAction Note ] Ch4. Connecting ActiveMQ - Connecting to ActiveMQ over the network

標籤

2015年8月13日 星期四

[ InAction Note ] Ch4. Connecting ActiveMQ - Connecting to ActiveMQ over the network

Preface:
The most common usage scenario is to run ActiveMQ as a standalone Java application. This implies that clients (producer and consumer applications) will use some of the network protocols to access the broker’s destinations. In this section, we’ll describe available network protocols you can use to achieve client-to-broker communication.

We’ll start with default TCP connector, which is most widely used and provides optimal performance. Next we’ll dig into the NIO connector, which also uses TCP network protocol underneath, but additionally provides a bit better scalability than TCP connector since it uses the NIO Java API. The UDP network protocol is often used on the internet, so UDP connector is next on our list. UDP protocol introduces some performance advantages, sacrificing reliability compared to the TCP protocol. The same applies to appropriate ActiveMQ connectors, so a UDP connector can offer some performance advantages over the TCP connector, but it’s still not often used because of the unreliability it introduces (as explained in more detail later). The SSL connector can be used to establish a secure connection to the broker, and finally we’ll show you how to communicate with the broker using HTTP. Of course, in every section we’ll discuss the pros and cons of every protocol. Table 4.1 contains a summarization of the connectors with a brief description.


Transmission Control Protocol (TCP)
Transmission Control Protocol (TCP) is today probably as important to humans as electricity. As one of the fundamental internet protocols, we use it for almost all of our online communication. It’s used as an underlying network protocol for a wide range of internet services such as email and the web, for example.

Hopefully you are already familiar with the basics of TCP, but let’s start our discussion of TCP by quoting from the specification, RFC 793 (http://mng.bz/Bns2):
The Transmission Control Protocol (TCP) is intended for use as a highly reliable host-to-host protocol between hosts in packet-switched computer communication networks, and in interconnected systems of such networks.

Since the broker and client applications are network hosts trying to communicate in a reliable manner, it’s easy to see why TCP is an ideal network protocol for a JMS implementation. So it shouldn’t come as a surprise that the TCP transport connector is the most frequently used ActiveMQ connector.

Before exchanging messages over the network, we need to serialize them to a suitable form. Messages must be serialized in and out of a byte sequence to be sent over the wire using what’s known as a wire protocol. The default wire protocol used in ActiveMQ is called OpenWire. The protocol specification can be found on the ActiveMQ website (http://mng.bz/u2eT). The OpenWire protocol isn’t specific to the TCP network transport and can be used with other network protocols. Its main purpose is to be efficient and allow fast exchange of messages over the network. Furthermore, a standardized and open protocol such as OpenWire allows native ActiveMQ clients to be developed for various programming environments. This topic and a description of other wire level protocols available for ActiveMQ are covered in chapter 9.

A default broker configuration starts the TCP transport listening for client connections on port 61616. The TCP connector URI uses the following syntax:
tcp://hostname:port?key=value&key=value

Please note that the bold portion of the URI denotes the required part. Any key/value pairs to the right of the question mark are optional and separated by an ampersand. We won’t discuss all transport options for appropriate protocols in this section or the sections that follow. This kind of material is best presented via the online reference pages. An up-to-date reference for the TCP connector can be found on the ActiveMQ website (http://mng.bz/ngU2).

The following configuration snippet provides an example of using the TCP connector in the ActiveMQ configuration file:
  1.   
  2.     "tcp" uri="tcp://localhost:61616?trace=true"/>  
  •   Note that the trace option has been added to the transport connector URI. This option instructs the broker to log all commands sent over this connector and can be helpful for debugging purposes. We have it here as an example of a transport tuning feature using a transport option. For more information on using the trace option for debugging, seechapter 14.

    The previous section outlined the use of this protocol in the client applications to connect to the broker. Just for reference, the following example shows how to run the consumer using the TCP transport connector:
    # mvn exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Consumer -Dexec.args="tcp://localhost:61616 CSCO ORCL"
    ...
    ORCL 65.71 65.78 up
    ORCL 66.07 66.14 up
    ORCL 65.93 65.99 down
    CSCO 23.30 23.33 up
    ...

    Some of the benefits of the TCP transport connector include the following:
    - Efficiency
    Since this connector uses the OpenWire protocol to convert messages to a stream of bytes (and back), it’s very efficient in terms of network
    usage and performance.

    - Availability
    TCP is one of the most widespread network protocols and has been supported in Java from the early days, so it’s almost certainly supported on your platform of choice.

    - Reliability
    The TCP protocol ensures that messages won’t be lost on the network (due to glitches, for example).

    New I/O API protocol (NIO)
    The New I/O (NIO) API was introduced in Java SE 1.4 to supplement the existing (standard) I/O API used in Java until then. Despite the prefix new in its name, NIO was never meant to be a replacement for the traditional Java I/O API. Its purpose was to provide an alternative approach to network programming and access to some low-level I/O operations of modern operating systems. The most prominent features of NIO are selectors and nonblocking I/O programming, allowing developers to use the same resources to handle more network clients and generally heavier loads on their servers.

    From a client perspective, the NIO transport connector is practically the same as the standard TCP connector, in terms of its use of TCP as the underlying network protocol and OpenWire as the message serialization protocol. The only difference is under the covers with the implementation of the transport, where the NIO transport connector is implemented using the NIO API. This makes the NIO transport connector more suitable in situations where
    - You have a large number of clients you want to connect to the broker
    Generally, the number of clients that can connect to the broker is limited by the number of threads supported by the operating system. Since the NIO connector implementation starts fewer threads per client than the TCP connector, you should consider using NIO in case TCP doesn’t meet your needs.

    - You have a heavy network traffic to the broker
    Again, the NIO connector generally offers better performance than the TCP connector (in terms of using less resources on the broker side), so you can consider using it when you find that the TCP connector doesn’t meet your needs.

    At this point it’s important to note that performance tuning of ActiveMQ isn’t just related to choosing the right connector. Many other aspects of ActiveMQ can be tuned, including the use of a network of brokers topology (see chapter 10) and setting various options for brokers, producers, and consumers (see chapter 13).

    The URI syntax for the NIO connector is practically the same as that of the TCP connector URI syntax:
    nio://hostname:port?key=value

    Now take a look at the configuration snippet:
    - Listing 4.3 Configuring the NIO transport connector
    1.   
    2.     ...  
    3.     
    4.          name="nio"  
    5.          uri="nio://localhost:61618?trace=true" />  
      Now run the stock portfolio example, but this time you’ll connect the publisher and consumer using different transport connectors. As figure 4.3 shows, the publisher will send messages using the NIO transport connector, whereas the consumer will receive those messages using the TCP transport connector.


    To achieve this, the stock portfolio publisher should be run using the following command:
    $ mvn exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Publisher -Dexec.args="nio://localhost:61618 CSCO ORCL"
    ...
    Sending: {price=65.713356601409, stock=JAVA, offer=65.779069958011, up=true}
    on destination: topic://STOCKS.JAVA

    Note that the nio scheme is used in the connection URI to specify the NIO connector. The consumer should use the TCP connector as shown below:
    $ mvn exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Consumer -Dexec.args="tcp://localhost:61616 CSCO ORCL"
    ...
    ORCL 65.71 65.78 up
    ORCL 66.07 66.14 up
    ORCL 65.93 65.99 down
    CSCO 23.30 23.33 up
    ...

    After both the consumer and producer are started, you’ll notice that messages are exchanged between applications as expected. The fact that they are using different connectors to communicate with the broker plays no role in this exchange.

    User Datagram Protocol (UDP)
    User Datagram Protocol (UDP) along with TCP make up the core of internet protocols. The purpose of these two protocols is identical—to send and receive data packets (datagrams) over the network. But there are two main differences between them:
    - TCP is a stream-oriented protocol
    It means that the order of data packets is guaranteed. There’s no chance for data packets to be duplicated or arrive out of order. UDP, on the other hand, doesn’t guarantee packet ordering, so a receiver can expect data packets to be duplicated or arrive out of order.

    - TCP also guarantees reliability of packet delivery
    It meaning that packets won’t be lost during the transport. This is ensured by maintaining an active connection between the sender and receiver. On the contrary, UDP is a connectionless protocol, so it can’t make such guarantees.

    As a result of these differences, TCP is used in applications that require reliability (such as email), whereas UDP usually finds it place in applications that require fast data transfers and can handle occasional packet loss (such as VoIP or online gaming).

    You can use the UDP protocol to connect to ActiveMQ by using the UDP transport connector. The URI syntax as below:
    udp://hostname:port?key=value

    The complete reference of the UDP protocol can be found at the ActiveMQ website (http://mng.bz/1i4g).


    Now let’s configure ActiveMQ to use both TCP and UDP transports on different ports. Here’s an example of such a configuration:
    1.   
    2.     ...  
    3.     "udp" uri="udp://localhost:61619?trace=true" />  
  •   To run a stock portfolio publisher using the UDP protocol, use the following command:
    $ mvn exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Publisher -Dexec.args="udp://localhost:61619 CSCO ORCL"
    ...
    Sending: {price=65.713356601409, stock=JAVA, offer=65.779069958011, up=true}
    on destination: topic://STOCKS.JAVA
    ...

    Secure Sockets Layer Protocol (SSL)
    Imagine yourself in a situation where you need to expose the broker over an unsecured network and you need data privacy. The same requirement emerged when the web outgrew its academic roots and was considered for corporate usage. Sending plain data over TCP became unacceptable, and a solution had to be found. The solution for secure data transfers was the Secure Sockets Layer (SSL), a protocol designed to transmit encrypted data over the TCP network protocol. It uses a pair of keys (one private and one public) to ensure a secure communication channel. ActiveMQ provides the SSL transport connector, which adds an SSL layer over the TCP communication channel, providing encrypted communication between brokers and clients. As always with SSL, keys and certificates are involved in configuring it, so this section is longer, as we’ll dig into this configuration in detail.

    The URI syntax for this protocol is:
    ssl://hostname:port?key=value

    Since the SSL transport is based on the TCP transport, configuration options are the same. More information for using the SSL connector is available on the ActiveMQ website (http://mng.bz/s8I2).

    ActiveMQ uses the Java Secure Socket Extension (JSSE) to implement its SSL functionality. Since its detailed description is out of the scope of this book, please refer to the online information for JSSE (http://mng.bz/7TYe) before proceeding to the rest of this section.

    To configure the ActiveMQ broker to use the SSL transport, the first thing to do is configure the ActiveMQ SSL transport. Change the  element in the${ACTIVEMQ_HOME}/conf/activemq.xml file as shown:
    1.   
    2.     "ssl" uri="ssl://localhost:61617?trace=true" />  
  •   But the SSL transport needs a few more items in order to work properly. Such required items include SSL certificates for successful SSL communication. Basically, JSSE defines two types of files for storing keys and certificates. The first are so-called keystores, which hold your own private certificates with their corresponding private keys. Trusted certificates of other entities (applications) are stored in truststores. To actually get the SSL transport working properly, the additional required items are discussed in detail in the next two sections.

    (For more information on configuring SSL, see chapter 6.)

    Now that you understand all the necessary elements needed for successful SSL communication, it’s time to connect to the secured transport. First, you’ll connect to the broker that’s secured with its default certificate. Next, you’ll walk through the procedure for creating your own certificates and running a broker and clients with them.

    USING SSL
    First a small experiment to see what happens when you try to connect to a broker using SSL and without providing any other SSL-related parameters. Connecting the stock portfolio consumer by changing the transport to use SSL without creating the proper stores will cause errors. Here’s an example of simply changing the transport:
    $ mvn exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Consumer -Dexec.args="ssl://localhost:61617 CSCO ORCL"

    Without creating and denoting the proper keystore and truststore, you can expect to see the following exceptions:


    Also, in the broker’s log you’ll see the following error:


    These errors mean that the SSL connection couldn’t be established. This is a generic error all clients will receive when trying to connect to the untrusted broker (without the proper keystore and truststore).

    When using JSSE, you must provide some SSL parameters using the appropriate system properties. In order to successfully connect to the broker via SSL, we must provide the keystore, the keystore password, and the truststore to be used. This is accomplished using the following system properties:
    * javax.net.ssl.keyStore—Defines which keystore the client should use
    * javax.net.ssl.keyStorePassword—Defines an appropriate password for the keystore
    * javax.net.ssl.trustStore—Defines an appropriate truststore the client should use

    Now take a look at the following example of starting the stock portfolio publisher using the default client certificate stores distributed with ActiveMQ:
    $ mvn \
    -Djavax.net.ssl.keyStore=${ACTIVEMQ_HOME}/conf/client.ks \
    -Djavax.net.ssl.keyStorePassword=password \
    -Djavax.net.ssl.trustStore=${ACTIVEMQ_HOME}/conf/client.ts \

    exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Publisher \
    -Dexec.args="ssl://localhost:61620 CSCO ORCL"

    ...
    Sending: {price=65.713356601409, stock=JAVA, offer=65.779069958011, up=true}
    on destination: topic://STOCKS.JAVA
    Sending: {price=66.071605671946, stock=JAVA, offer=66.137677277617, up=true}
    on destination: topic://STOCKS.JAVA
    Sending: {price=65.929035001620, stock=JAVA, offer=65.994964036622, up=false}
    on destination: topic://STOCKS.JAVA
    ...

    Note the use of the JSSE system properties in bold. These properties provide the necessary keystore, keystore password, and truststore. After providing these necessary SSL-related parameters, the publisher will connect successfully to the broker as intended without error. Of course, if the client isn’t located on the same computer as your broker, you’ll need to copy these files and adapt the paths appropriately.

    Similarly, the consumer can be run using the following command:
    $ mvn \
    -Djavax.net.ssl.keyStore=${ACTIVEMQ_HOME}/conf/client.ks \
    -Djavax.net.ssl.keyStorePassword=password \
    -Djavax.net.ssl.trustStore=${ACTIVEMQ_HOME}/conf/client.ts \

    exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Consumer \
    -Dexec.args="ssl://localhost:61620 CSCO ORCL"

    ...
    ORCL 65.71 65.78 up
    ORCL 66.07 66.14 up
    ...

    Again, note the use of the JSSE system properties in bold. Now both clients can communicate with the broker using the encrypted network channels provided by the SSL transport connector. Working with the default certificate, keystore, and truststore is okay for development purposes, but for a production system, it’s highly recommended that you create and use your own certificates. You can even disable ciphers that you may not be using. In most cases, you’ll need to purchase an appropriate SSL certificate from the trusted certificate authority.

    CREATING YOUR OWN SSL RESOURCES
    For development purposes, you’ll want to create your own self-signed certificates. The rest of this section will lead you through the process of creating and sharing selfsigned certificates. For that purpose the keytool will be used—the command-line tool for managing keystores that’s distributed with Java.

    First, you must create a keystore and a certificate for the broker. Here’s an example of this using the keytool that comes with the JDK:


    The keytool application prompts you to enter certificate data and create a keystore with the certificate in it. In this case we’ve created a keystore file named mybroker.ks with the password test123. The next step is to export this certificate from the keystore, so it can be shared with the broker’s clients:
    # keytool -export -alias broker -keystore mybroker.ks -file mybroker_cert
    輸入 keystore 密碼: test123
    認證儲存在檔案

    This step creates a file named mybroker_cert, containing a broker certificate. Now you must create a client keystore with the appropriate certificate using a command similar to the one that was used previously to create the broker’s keystore:


    The result of this command is the myclient.ks file with the appropriate certificate for the client side. Finally, the client truststore must be created and the broker’s certificate must be imported into it. Again, keytool is used to achieve this with the following command:


    With this step, all the necessary stores were created and the broker certificate was imported into the keystore. Now the stock portfolio example can use them. Remember to start the broker using the newly created certificate. One way to do this is to replace the default keystore files and the broker cert in the conf directory with the ones that were just created.

    Another way is to pass the SSL-related system properties to the command used to start our broker. For that we need first to copy certificates with their original names (with prefix my in the name) to the conf/ directory:
    # cp mybroker.ks ${ACTIVEMQ_HOME}/conf/
    # cp myclient.ks ${ACTIVEMQ_HOME}/conf/
    # cp myclient.ts ${ACTIVEMQ_HOME}/conf/

    So now we can pass the system property and use a keystore other than default one:
    ${ACTIVEMQ_HOME}/bin/activemq console \
    -Djavax.net.ssl.keyStorePassword=test123 \
    -Djavax.net.ssl.keyStore=${ACTIVEMQ_HOME}/conf/mybroker.ks

    Finally, we can achieve the same thing with the  element in the ActiveMQ configuration file, as shown here:
    1. "http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">  
    2.     ...  
    3.       
    4.         "file:${activemq.base}/conf/mybroker.ks" keyStorePassword="test123" />  
  •     
  •   
  •     ...  
  •   
  • Now let’s see how to reflect these same changes to the clients. If you try to run the client applications with the old certificate file, you’ll get the unknown_certificate exception, just as when the client attempted to access the broker without using any certificate. So you’ll have to update the command like the following:
    $ mvn \
    -Djavax.net.ssl.keyStore=${ACTIVEMQ_HOME}/conf/myclient.ks \
    -Djavax.net.ssl.keyStorePassword=test123 \
    -Djavax.net.ssl.trustStore=${ACTIVEMQ_HOME}/conf/myclient.ts \
    exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Publisher \
    -Dexec.args="ssl://localhost:61617 CSCO ORCL"

    ...
    Sending: {price=65.713356601409, stock=JAVA, offer=65.779069958011, up=true}
    on destination: topic://STOCKS.JAVA
    Sending: {price=66.071605671946, stock=JAVA, offer=66.137677277617, up=true}
    on destination: topic://STOCKS.JAVA
    Sending: {price=65.929035001620, stock=JAVA, offer=65.994964036622, up=false}
    on destination: topic://STOCKS.JAVA
    ...

    ENABLING AND DISABLING SSL CIPHERS
    The SSL cipher suites for the ActiveMQ SSL transport are provided by the JVM. For specific information about these cipher suites, see the documentation on the Sun JSSE provider (http://mng.bz/7TYe). The Sun JSSE provider supports a long list of cipher suites, and these are utilized in their default preference order. In some situations, there can be a need to disable certain ciphers. Examples of such situations include the discovery of a vulnerability in a particular cipher or a requirement to support only certain ciphers. To make it easy to enable/disable cipher suites, starting in ActiveMQ 5.4.0, a new option for the SSL transport named transport.enabledCipherSuites is available. Here’s an example of this new option:
    1.   
    2.   
    3.    name="ssl"   
    4.    uri="ssl://localhost:61617?transport.enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA" />  
      In the preceding example, the SSL_RSA_WITH_RC4_128_SHA cipher suite is the only one that’s been enabled on the ActiveMQ SSL transport. Additional cipher suites can be enabled using a comma-separated list. The purpose of this new option is for added security as it allows only certain cipher suites to be enabled. This can be handy in environments that consider some cipher suites too weak to leave them enabled, such as the Payment Card Industry (PCI).

    Hypertext Transfer Protocol (HTTP/HTTPS)
    In many environments, firewalls are configured to allow only basic services such as web access and email. So how can ActiveMQ be used in such an environment? This is where the HTTP transport comes into play.

    Hypertext Transfer Protocol (HTTP) was originally designed to transmit hypertext (HTML) pages over the web. It uses TCP as an underlying network protocol and adds some additional logic for communication between browsers and web servers. After the first boom of the internet, web infrastructure and the HTTP protocol in particular found a new role in supporting web services, commonly used these days to exchange information between applications. The main difference is that in the case of web services, XML-formatted data is transmitted using the HTTP protocol rather than HTML data.

    ActiveMQ implements the HTTP transport connector, which provides for the exchange of XML-formatted messages with the broker using the HTTP protocol. This is what allows ActiveMQ to bypass strict firewall rules.

    The URI syntax of this transport connector is as follows:
    http://hostname:port?key=value

    Secure HTTP (HTTP over SSL or HTTPS) is also supported by this transport:
    https://hostname:port?key=value

    The transport connectors section of the XML configuration in this case looks similar to those used in previous sections:
    1.   
    2.     ...   
    3.     "http" uri="http://localhost:8080?trace=true" />  
  •   In order to run the clients using the HTTP transport protocol, one dependency must be added to the classpath. The HTTP transport is located in the ActiveMQ optional module, so you’ll have to add it to the application’s classpath (along with appropriate dependencies). Using Maven, you’ll need to add the following dependency to the pom.xml file:
    1.   
    2.     ...  
    3.     org.apache.activemq  
    4.     activemq-optional  
    5.     5.4.1  
  •     ...  
  •   This will include activemq-optional module and all its dependencies to the classpath. In case you don’t use Maven to manage your classpath, be sure to include all of these JARs into your classpath:
    $ACTIVEMQ_HOME/lib/optional/activemq-optional-.jar
    $ACTIVEMQ_HOME/lib/optional/commons-httpclient-.jar
    $ACTIVEMQ_HOME/lib/optional/xstream-.jar
    $ACTIVEMQ_HOME/lib/optional/xmlpull-.jar

    Finally, the stock portfolio publisher is ready to be run using the HTTP transport:
    $ mvn exec:java -Dexec.mainClass=org.apache.activemq.book.ch4.Publisher \
    -Dexec.args="http://localhost:8080 CSCO ORCL"

    ...
    Sending: {price=65.713356601409, stock=JAVA, offer=65.779069958011, up=true}
    on destination: topic://STOCKS.JAVA
    Sending: {price=66.071605671946, stock=JAVA, offer=66.137677277617, up=true}
    on destination: topic://STOCKS.JAVA
    Sending: {price=65.929035001620, stock=JAVA, offer=65.994964036622, up=false}
    on destination: topic://STOCKS.JAVA
    ...

    As stated previously, when using the HTTP transport, all broker-to-client communication is performed by sending XML messages. This type of communication can have an impact on the overall system performance compared to the use of the TCP transport with the OpenWire protocol (which is tuned specifically for messaging purposes). So if performance is a concern, you’re best to stick to the TCP transport and find some other workaround for the firewall issues.

    Supplement:
    ActiveMQ SSL Exchanges and Handshake Error Messages
    FAQ - Setting up the Key and Trust Stores
    Before starting the broker's VM set the SSL_OPTS enviorment variable so that it knows to use the broker keystore.
    # export SSL_OPTS = "-Djavax.net.ssl.keyStore=/path/to/broker.ks -Djavax.net.ssl.keyStorePassword=password"

    Java keytool 基本指令介紹
    1. 匯入憑證到keystore
    keytool –import –alias xxx –file xxx.cer –keystore .keystore

    2. 查詢keystore的內容
    keytool –list –v –keystore .keystore

    3. 刪除keystore內的其中一個憑證
    keytool –delete –alias xxx –keystore .keystore

    4. 產生金錀對(RSA為非對稱加密的演算法)
    keytool -genkey -alias xxx -keyalg RSA -keystore .keystore

    5. 產生憑證申請檔
    keytool -certreq -alias xxx -file certreq.txt -keystore .keystore

    6. 查詢PKCS12類型keystore的內容
    keytool –list –v –keystore .keystore -storetype pkcs12

    7. 建立一個含有私鑰的keystore
    keytool -genkey -alias keyAlias -keyalg RSA -keystore keystore.jks

    8. 修改keystore的密碼
    keytool -storepasswd -new newPassword -keystore keystore.jks


    沒有留言:

    張貼留言

    網誌存檔

    關於我自己

    我的相片
    Where there is a will, there is a way!