Store-and-forward processing in multiple messaging broker topologies

Overview

An application wishing to send or receive messages using the IBM® Reliable Transport Extension needs a connection to an active messaging broker for request processing to complete. Depending on the requirements of the scenario, a deployment may use a single messaging broker or multiple messaging brokers.

A single broker topology

Consider the following scenario, in which two applications are communicating via a single messaging broker:

In this scenario, app1 is placing a message for app2 on queue2 while app2 is placing a message for app1 on queue1. Both queue1 and queue2 are hosted by a single messaging broker, broker1, so both applications require connection configurations for this broker.

A multiple broker topology

If app1 and app2 are executing on separate machines, then either one or the other will have to make remote network connections to communicate with the messaging broker. In this situation, it may be desirable to add a second messaging broker to the topology so each application need only use a local network connection, as in the following diagram:

The broker most adjacent to each application is called the "home" broker. In the diagram, broker1 is the home broker for app1 and broker2 is the home broker for app2.

Each input queue for each application is hosted by the corresponding home broker. Provided the home broker is active, the application will be able to process the messages queued for it using the local network connection.

However, since each applications needs to send messages to the other's queue, they both need connections to the remote broker. This has two disadvantages:

• Each application now requires two separate broker connection configurations; one for its home broker to receive messages and one for the corresponding remote broker to send messages.
• If the connection to the remote broker is temporarily unavailable, an application will be unable to send messages to the other party.

A multiple broker topology with store-and-forward processing

Store-and-forward processing provides an alternative approach when there are multiple brokers in the topology. For example, consider the following diagram:

As before, each application has a home broker from which it receives messages. However, this time the applications do not make direct connections to the corresponding remote broker to send a message. Instead, the applications send a message to a special, store-and-forward queue on their home broker.

In the diagram, the store-and-forward queue for queue2 is labelled saf:queue2. When app1 wants to send a message to app2 it sends a message to saf:queue2 using its home broker connecion. Once the send operation has completed successfully, the delivery of the message to queue2 becomes the responsibility of the broker, broker1. When a connection to broker2 can be established, messages will flow from the store-and-forward queue to the target queue.

Deployment considerations

Store-and-forward queue names

Store-and-forward queues are identified by a naming scheme. If a store-and-forward queue addresses queueName over the broker connection configuration brokerName then its name will be:

storeAndForward:brokerName/queue/queueName

The messaging protocol generates this name automatically when the storeAndForward property is true so applications using the Connection API do not need to be concerned with this naming scheme. However, the names of the store-and-forward queues do need to be considered in some circumstances, for example:

• The configuration of a secured broker configuration will need to provide authorization rules for the store-and-forward queues.
• An application using the broker statistics API will be given details of the store-and-forward queues.
• An application using the external client API will need to generate the appropriate queue names to use store-and-forward processing. The brokerName element is limited to 20 characters and the queueName element can contain up to 79 characters.

Store-and-forward synchronization queue

A special store-and-forward synchronization queue is used to store meta data about the messages currently being flowed from remote brokers that are using store-and-forward processing. It is automatically created when a broker receives an incoming store-and-forward connection.

The store-and-forward synchronization queue is called storeAndForwardSyncQueue. The configuration of a secured broker will need to provide authorization rules to permit incoming connections from remote brokers that are using store-and-forward processing to access the synchronization queue.

Store-and-forward bridge connections

If an application wishes to use store-and-forward processing to send a message to a queue on a remote broker, its home broker will require a store-and-forward bridge connection configuration to allow it to contact the remote broker. A bridge connection configuration is specified in the same way as a broker connection configuration for an application but can have additional properties, such as the user credentials to use for the bridge connection.

For example, if a broker has a store-and-forward queue, storeAndForward:broker1/queue/queue1, it will attempt to connect to a broker using the connection configuration, broker1 and forward messages to the remote queue, queue1.

It is important to note that store-and-forward processing is not instantaneous. The time taken to deliver an individual message will depend on several factors including the availability of the remote broker connection and the number of messages on the store-and-forward queue. Once an application has placed a message on a store-and-forward queue, a number of events must occur before it will be delivered to the remote queue:

1. After a store-and-forward queue is first created, the bridge connection manager will automatically start a "pipe" to the remote broker and a "flow" to forward messages to the remote queue. This will happen within about a minute after the first message is placed on the queue.
2. Once a pipe has been started, it will attempt to connect to the remote broker. If the remote broker cannot be reached, then repeated attempts will be made every few seconds.
3. Once a pipe has been connected, the messages will begin to flow from the store-and-forward queue to the remote queue. Message order is preserved so the youngest message on the queue will have to wait for those placed on the queue before it to be delivered first.

Dynamic configuration for bridge connections

A messaging broker is a long running application which needs to remain active to provide messaging services to client applications. Changes to zero.config cannot be read and acted on without restarting the messaging broker and so potentially impacting client applications. However, it is possible to make changes to bridge connection configurations to allow store-and-forward processing to connect to new remote brokers without restarting the messaging broker.

To use the dynamic configuration mechanism, the bridge connection configurations must be placed in a separate configuration file. For example, if the bridge connection configurations are placed in the file config/brokers.config then the following lines in zero.config will enable dynamic configuration:

@include "brokers.config"
/config/messaging/broker/configFile = "config/brokers.config"

The bridge connection manager processes a dynamic configuration file as follows:

1. The last written time stamp of the dynamic configuration file specified by /config/messaging/broker/configFile is examined. If the time stamp has changed then dynamic configuration takes place:
   1. All running store-and-forward bridge connections are suspended.
   2. All messaging broker connection configurations stored under /config/connection/messaging are deleted.
   3. All messaging broker connections and connection pools are closed.
   4. The dynamic configuration file is read.
2. The bridge connection manager resumes normal processing and restarts the required bridge connections.

The following notes apply to the dynamic configuration mechanism:

• The dynamic configuration file is only supported in long running applications containing a running messaging broker.
• Dynamic configuration is only supported for messaging broker connections. The dynamic configuration file must only contain configuration keys of the form /config/connection/messaging/ brokerName
• The dynamic configuration file can use the @include directive to reference additional files but included files will not be monitored for changes. The dynamic configuration will only be loaded when the file specified by /config/messaging/broker/configFile has changed.
• Dynamic configuration will destroy all broker connection configurations including the home broker connection configuration that may be used by a messaging kicker in the same application. This can cause a temporary suspension of the messaging kicker functionality but the service will resume after the dynamic configuration has completed. However, it is important that the dynamic configuration file includes the home broker connection configuration to allow the messaging kicker to resume.