Enterprise Messaging in a Heterogeneous Environment

Introduction

I work for a small company that long ago decided to standardize its core application on the .NET platform. The resulting system was entirely proprietary, developed in house and based on Microsoft technology (for example, MSMQ, IIS, and SQL Server). As the business grew, new business processes were introduced and it made sense to seek the "help" of third-party applications to carry out steps within them. In particular, one of the crucial steps involved large document transformation. After a series of inquiries and trials, a powerful, dependable, and speedy Java-based transformation engine was purchased. In solving one problem, another is created: the integration of a Java application into a homogeneous, .NET-based SOA.

Integration can be accomplished when both .NET and Java messaging systems operate under the same message transportation standards, message structures are interoperable, and a platform neutral protocol is used to relay messages from one system to the other.

The following discusses how to use enterprise messaging to integrate Java and .NET applications into a single logical Enterprise Messaging Bus.

Transportation

Central to success with enterprise messaging is the establishment of how messages will be sent and how they will be received: "the transportation layer." When using queuing systems such as MSMQ for message transport, it's common that a sender, or "producer," of a message be required to know or discover which queue the receiver, or "consumer," of the message is connected to. If the producer expects a response from the consumer, the producer provides the location or alias of the queue it itself is connected to in the request message. The consumer then dynamically connects to the producer's queue and sends the reply. This type of architecture has two main problems:

  1. Security: Access must be configured for each producer/consumer interaction (one producer may communicate with "n" number of consumers). This can be a nightmare for any infrastructure/operations team to support.
  2. Coupling: Message producers are responsible for locating and connecting to one to many consumer queues. They are also responsible for message delivery to those queues. This has a tendency to further couple, rather than decouple, services. The goal of an SOA is for services to be loosely coupled.

A queuing architecture implemented as described above (often called "Point-to-Point") results in "spaghetification"—a web of integration points that must be maintained in a given environment. SOAs based on Web services also tend to follow this Point-to-Point pattern (until WS-Notification is generally accepted, that is).

Enter publish/subscribe. The basic premise of this idea is that a producer can send a message and know nothing about the location of the consumer. In fact, multiple consumers can receive and do work on the same message. Consumers, or "subscribers," profess their interest in a message and are notified when a message they have asked to receive is sent. Messages, in turn, must be filterable so the messaging system can properly deliver them. This filterable attribute is akin to the Label property of a System.Messaging.Message object in .NET or the Topic for a JMS message in Java. But, the generic term I'd like to use is TIBCO's word for the concept, "subject."

In a publish/subscribe architecture, subject naming conventions are paramount. Generally speaking, a good naming convention is arranged from domain downward. For example, suppose the following was the convention:

WORLD.CONTINENT.COUNTRY.STATE.COUNTY.CITY

In this example, a subscriber could feasibly listen for any messages destined for earth by subscribing to messages with the subject "EARTH.*" (note the wildcard character "*"). Alternatively, if a subscriber were more interested in messages destined for a specific continent on earth, it could apply a subject filter such as "EARTH.AUSTRALIA.*." In this example, all messages destined for the Australian continent on planet Earth would be received by the subscriber (this type of wildcard subscription is useful when creating diagnostic or message routing applications). The analogy continues to the very lowest level, where you can subscribe to messages destined only for a particular city (and breaks down once you realize not all countries have states or the concept of counties).

For practical purposes, the following convention can be employed:

COMPANY.DIVISION.ROLE.ENTITY.Action:ACME.TELECOM.PROVISIONING.DSL.Activate

Queuing systems such as MSMQ can be modified to simulate publish/subscribe through the introduction of a custom message "router." At startup, each subscriber registers its interest in one to many subjects. Registration includes the subscriber's local queue address. It is then the router's responsibility to deliver messages to registered subscriber queues. The following diagram illustrates how the router maintains correlation between subjects and queues:

In this model, all publishers send messages to a single central queue. Each message sent to this queue is filtered by the router based on subject and forwarded to the subscribers interested in the message.

The advantages to this model are the following:

  1. The router is the only component requiring access to consumer queues.
  2. Queues can be redeployed and configured with little or no impact to other services.
  3. The architecture is scalable and loosely coupled—message producers need only connect to and send messages to one queue. The messaging infrastructure is responsible for delivering messages to consumers.

The problem with this model, however, is that the router becomes a bottleneck for the messaging architecture because all messages funnel through it. To overcome this, multiple instances of the router should run concurrently to distribute the workload.

Once services on the .NET side of the house are operating under a pseudo publish/subscribe messaging paradigm, it's easy to extend the metaphor to Java-based messaging systems such as JMS, which support publish/subscribe out of the box.

Subscribing to Messages

MSMQ and JMS both provide mechanisms for event-based notification. That is, consumers can be notified when messages are sent to them by the messaging infrastructure.

In C#, the PeekCompleted event is fired for the first message in the queue (and is continuously fired until the message is removed or its lifespan expires). The following code sample wires an event handler to the PeekCompleted event and interrogates the message's Label property to determine whether it matches the subject name the consumer is interested in.

private MessageQueue
   m_messageQueue;

private string
   m_subject;

public Subscriber(string queueName, string subject)
{
   m_subject                          = subject;
   m_messageQueue                     = new MessageQueue(queueName);
   MessageQueue.EnableConnectionCache = false;

   m_messageQueue.PeekCompleted       += new
      PeekCompletedEventHandler(OnPeekCompleted);
}

protected void OnPeekCompleted(object sender,
                               PeekCompletedEventArgs asyncResult)
{
   try
   {
      //end the peek and get the message
      using(Message message = m_messageQueue.EndPeek(asyncResult))
      {
         //filter on label. If we are doing a wildcard subscription,
         //check if the message label is a partial match to the
         //subject; otherwise, check for an absolute match.

         if(m_subject.IndexOf("*") > -1)
         {
            //We're doing a wildcard subscription here
            if(message.Label.IndexOf(m_subject.Replace("*",null)) > -1)
            {
               //Pull the message off the queue
               m_messageQueue.ReceiveById(message.Id);
            }
            else return;
         }
         else if(m_subject.Equals(message.Label))
         {
            //Pull the message off the queue
            m_messageQueue.ReceiveById(message.Id);
         }
         else return;

         //Execute business logic
         ...
      }
   }
   finally
   {
      //Resume listening for messages
      m_messageQueue.BeginPeek();
   }
}

In Java, a subscriber must implement the javax.jms.MessageListener interface and define a method called onMessage(). The JMS subscriber connects to the JMS Provider that automatically performs the subject filtering that has to be done explicitly by the C# subscriber. onMessage() is invoked, then, only when a message is published on the subject the subscriber is interested in. JMS also has out-of-the-box support for wildcard subscription ("*").

private TopicConnection
   m_connection;

private TopicSession
   m_session;

private TopicConnectionFactory
   m_factory;

public Subscriber(String subject, String broker, String connectID,
                  String userID, String password)
                 throws javax.jms.JMSException{

   try{

      //Initialize the session and connection with the JMS Provider
      Hashtable props = new Hashtable();
      props.put(Context.INITIAL_CONTEXT_FACTORY,
                org.exolab.jms.jndi.rmi.
                RmiJndiInitialContextFactory.class.getName());
      props.put(Context.PROVIDER_URL, broker);
      Context context = new InitialContext(props);

      m_factory = ((TopicConnectionFactory)
                  context.lookup("JmsTopicConnectionFactory"));

      if(null == m_factory)
      {
         throw new RuntimeException("Failed to locate connection
                                     factory");
      }

      m_connection = m_factory.createTopicConnection(userID, password);
      m_session    = m_connection.createTopicSession(false,Session.
                                                     AUTO_ACKNOWLEDGE);

      //Set up a subscriber
      m_session.createSubscriber(m_session.createTopic(m_subject)).
         setMessageListener(this);
      m_connection.start();
   }
   catch(javax.naming.NamingException nfe){

      throw new RuntimeException(nfe.getMessage());
   }
}

public void onMessage(javax.jms.Message message){

   try{

      //Business Logic
      ...
   }
   catch(javax.jms.JMSException jme){

      //Handle the exception
      ...
   }
}

Enterprise Messaging in a Heterogeneous Environment

Message Structure

Many homogenous systems using enterprise messaging support the sending of complex objects between services using binary streams. When producers and consumers are written in the same programming language, the binary streaming of complex objects works because there are no type compatibility barriers. For example, suppose you created a .NET BroadbandOrder class with properties such as CustomerName, Address, OrderNumber, and so forth. When a .NET producer publishes this object (the .NET MSMQ API can be configured to use binary serialization when sending complex objects), the object is deserialized and reconstructed when a .NET consumer receives it—the consumer and producer's binary serializers are compatible. If you created the same BroadbandOrder class in Java and were somehow able to send the binary stream from a .NET producer to a Java consumer, the Java consumer would not be able to deserialize and reconstruct the BroadbandOrder object because Java and .NET's binary serializers are not compatible.

Where binary serialization fails, XML serialization succeeds. In short, XML serialization is the process of converting a complex object into an XML structure and converting back from an XML structure into a complex object. Both .NET and Java have tools and APIs that can be used to facilitate XML serialization. The Visual Studio 2003 IDE comes with a program called .xsd.exe. which can create a complex object in C# from an XSD and vice versa. Java has a multitude of programs that accomplish the same tasks. The WebMethods Glue utility (formerly called Electric XML from the Mind Electric, Inc.) is a good Java XML serializing tool. It comes with a script called "schema2Java.bat" that creates a complex object in Java from an XSD. Both XML serialization utilities create classes for each complexType in a given XSD. Each attribute or element within a complexType is then represented as a property or public field within the class.

So, if you continue with the broadband order example and wanted to try again to send the broadband order from a .NET producer to a Java consumer, the broadband order must first be represented in XSD.

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <xs:element name="BroadbandOrder">
      <xs:complexType>
         <xs:sequence>
            <xs:element name="OrderNumber" type="xs:string"/>
            <xs:element name="DateSubmitted" type="xs:dateTime"/>
            <xs:element name="Service" ref="Service"/>
            <xs:element name="BillToAddress" ref="Address"/>
            <xs:element name="ShipToAddress" ref="Address"/>
         </xs:sequence>
      </xs:complexType>
   </xs:element>
   <xs:element name="Address">
      <xs:complexType>
         <xs:sequence>
            <xs:element name="AddressLine1" type="xs:string"/>
            <xs:element name="AddressLine2" type="xs:string"/>
            <xs:element name="City" type="xs:string"/>
            <xs:element name="State" type="xs:string"/>
            <xs:element name="Zip" type="xs:string"/>
      </xs:sequence>
      </xs:complexType>
   </xs:element>
   <xs:element name="Service">
      <xs:complexType>
         <xs:sequence>
            <xs:element name="ServiceTypeID" type="xs:int"/>
            <xs:element name="ServiceName" type="xs:string"/>
         </xs:sequence>
      </xs:complexType>
   </xs:element>
</xs:schema>

Using XML serialization tools, two BroadbandOrder complex objects, or "data classes," are then produced, one in C#:

@echo off
rem generates .NET class(es) based on an XSD

set PATH=%PATH%;C:\Program Files\Microsoft Visual Studio .NET 2003\
                   SDK\v1.1\Bin
set XSD_HOME="C:\Solutions\MTM\XSD"
set XSD=BroadbandOrder.xsd
set OUT_DIR="C:\Solutions\MTM\MiddleTier\DataClass"
set NAMESPACE=Acme.Telecom.OrderManagement

echo Generating .NET Data classes
xsd.exe %XSD_HOME%\%XSD% /classes /namespace:%NAMESPACE% /out:%OUT_DIR%
echo Done.

and the other in Java:

@echo off
rem generates Java Data classes based on an XSD
set BUILD_HOME="C:\Solutions\OrderManager\Java\bin"
set PATH=%PATH%;C:\java\TME\exml\bin;
set XSD_HOME="C:\Solutions\XSD"
set XSD=BroadbandOrder.xsd
set PACKAGE=com.acme.telecom.orderManagement
set OUT_DIR="C:\Solutions\OrderManager\Java\src\com\acme\telecom\
                orderManagement"

echo Generating Java data classes
cd %OUT_DIR%
copy %XSD_HOME%\%XSD% %OUT_DIR%
schema2java %BRIDGE_XSD% -p %PACKAGE% 
echo Done.

The .NET producer leverages the System.Xml.Serialization.XmlSerializer class to convert the broadband order data class into a System.String.

XMLSerializer
   dataSerializer  = new XmlSerializer(typeof(BroadbandOrder));

StringBuilder 
   xmlBuilder      = new StringBuilder();

StringWriter
   xmlWriter       = new StringWriter(xmlBuilder);

dataSerializer.Serialize(xmlWriter, data);

return xmlBuilder.ToString();

The Java consumer, after receiving the message in java.lang.String format (System.String and java.lang.String types are generally compatible), deserializes the XML string into its version of the broadband order data class using electric.xml.Document and electric.xml.io.IReader objects from the Glue API.

Document doc     = new Document(message);
IReader reader   = new LiteralReader(doc);

return (BroadbandOrder_TYPE)reader.readObject(BroadbandOrder_TYPE.class);

The point of all of this is that, for enterprise messaging in a heterogeneous environment to succeed, messages, or contracts, should be represented as XSDs (and stored in a repository of some description). At runtime, because XML serializers in .NET and Java are compatible, Java and .NET services can exchange XML messages and deserialize them into useable complex objects or data classes that are meaningful in their respective languages.

Synchronous Messaging

Synchronous messaging is the act of sending a message then waiting, or blocking, until a response message is received. In most cases, it is a good idea for services to respond to the requesting application with a business-level message (in other words, not just an acknowledgement) even when no "business data" is generated by the service. This is because many SOAs make use of orchestration or workflow engines to manage business processes. Processes often make use of multiple services during execution and incorporate escalation tasks into their design to deal with either the lack of a response from a service (timeout) or an error response (service was not able to process the request due to an exception). Synchronous messaging therefore becomes crucial for proper error handling and escalation.

Producers, then, need to correlate requests sent to consumers with responses sent from them.

JMS supports synchronous publish/subscribe messaging out of the box. Producers create a new javax.jms.TopicRequestor object. A call to request() on this object sends the message, and then blocks until the response is received or a timeout occurs. This API dynamically builds a Topic that is unique to the request, so a response message from a consumer using this dynamic Topic automatically correlates the response to the request. JMS consumers can retrieve this dynamic Topic through a call to getJMSReplyTo() on the javax.jms.Message request message object.

MSMQ, on the other hand, which does not support publish/subscribe out of the box, does not have built-in synchronous, "label-based" messaging support. The System.Messaging.Message class contains a property called Extension that holds a binary array holding application-specific data. .NET producers can make use of this property to indicate to consumers which Label they expect the response to be published with.

Adding the response label to the Extension property:

string
   responseLabel = ACME.TELECOM.PROVISIONING.DSL.Response.
                   98fc657a-335c-46d6-abfb-a763c09ced71;

Message
   message = new Message(messageBody);

//Add the response label to the Extension property
message.Extension = System.Text.Encoding.ASCII.GetBytes(responseLabel);

Getting the response label from the Extension property:

string
   responseLabel = System.Text.Encoding.ASCII.GetString(message.
                                                        Extension);

Enterprise Messaging in a Heterogeneous Environment

Bridging the Gap

Web Services offer an attractive solution for message exchange between .NET and Java applications. They support protocols and standards such as HTTP and SOAP and maintain the integrity of strongly typed data (in other words, xsd:string maps to System.String in .NET and java.lang.String in Java). A Web service-based bridge is the key ingredient to JMS and MSMQ interoperability once the architecture has standardized on an interoperable message structure.

The resulting solution consists of four components that, together, are called the "Message Bridge." The Message Bridge essentially allows .NET applications to publish messages to a JMS Provider (OpenJMS in this case), and Java applications to publish messages to MSMQ.

Components:

  • JMS Message Broker—Subscribes to messages published to a JMS Provider and routes them to the .NET Web Service.
  • MSMQ Message Broker—Subscribes to messages published to an MSMQ queue and routes them to the Java Message Service.
  • Java Message Service—Web service that exposes a Web method called routeMessage() that publishes a message to the JMS Provider.
  • .NET Message Service—Web service that exposes a Web method called RouteMessage() that publishes a message to an MSMQ queue.

The following diagram illustrates how the Message Bridge components interact when a message is published from a .NET service to a Java service and back from the Java service to the .NET service:

The end goal of the Message Bridge is to hide the fact that it exists from all services, making integration appear seamless. The resulting logical architecture incorporates a single integration point, the Enterprise Message Bus, for all services, be they .NET or Java based. Services need only connect to the "central" MSMQ or a JMS provider to publish messages to any other service in the architecture. Because messages are standardized on XML, the distinction between messages produced by .NET or Java services does not exist—the producer has no idea what programming language the consumer is written in.

Understanding the Message Bridge Web Services

The Message Bridge Web Services have one purpose: the routing of messages to their respective messaging systems. Each defines only one Web method, routeMessage(String message) on the Java side and RouteMessage(string message) on the .NET side (for reasons discussed later, the arguments must be strings instead of complex types). Each Web service is expecting an XML message with the following structure:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <xs:element name="BridgeMessage">
      <xs:complexType>
         <xs:sequence>
            <xs:element name="Subject" type="xs:string" />
            <xs:element name="ReplySubject" type="xs:string" />
            <xs:element name="Data" type="xs:string" />
         </xs:sequence>
      </xs:complexType>
   </xs:element>
</xs:schema>

Upon invocation, the Message Bridge Web services deserialize the BridgeMessage into a data class; then use the properties of the data class to forward the message to the messaging system.

The "Data" node of the BridgeMessage contains the request or response message (XML string) either going to or from a consumer. The "Subject" node is used by the Web services to map a Label to a Topic. So, on the .NET side, the "Subject" becomes the message Label, and on the Java side it becomes the Topic. The "ReplySubject" node (optional) is bound by the Java Message Service to the JMS message through setJmsReplyTo(ReplySubject). It is added to the Extension property of the MSMQ message by the .NET Message Service. After mapping the BridgeMessage to platform-specific message objects, the message is forwarded by the .NET Message Service to MSMQ, or by the Java Message Service to JMS, depending on which is invoked.

The Java Message Service leverages Axis (the Apache Web Services project) is deployed on Apache Tomcat. Axis provides a library ideal for deploying Web services and creating proxy classes for Web service clients (check out the Axis Home Page for details). The .NET Message Service is a standard .NET Web service created in Visual Studio .NET and is hosted on an IIS server.

Understanding the Message Bridge Message Brokers

The message brokers invoke the Web methods of the Message Bridge Web services. They are responsible for:

  • Determining whether a message should be sent over the bridge.
  • Connecting to the Message Bridge Web Services.
  • Constructing and serializing the BridgeMessage data class.

Subject naming standardization is crucial for the Message Bridge to function correctly. The JMS Message Broker and MSMQ Message Broker route select messages based on subject. The original idea was that all messages would be routed over the bridge, but if this occurred, a single message would be endlessly passed back and forth. So, each message broker must be configured with a list of subject names that can be sent over the bridge. Both message brokers receive all messages in their respective "domains" (the MSMQ Message Broker subscribes to all MSMQ messages, and the JMS Message Broker subscribes to all JMS messages) and subsequently filter all messages.

Next, the message brokers make use of Web service proxy classes to connect to and invoke the Message Bridge web services. The details of connecting to a Web service and constructing and deconstructing SOAP requests are hidden by the proxy classes. These classes are automatically generated by utilities that interrogate the Message Bridge web service WSDLs. The Java Message Broker uses Axis for proxy class generation. Axis comes with an ant task called "axis-wsdl2java" that interprets the .NET Message Service's WSDL and creates Java proxy classes. The .NET Message broker uses "wsdl.exe," part of Visual Studio 2003, to interpret the Java Message Service's WSDL and creates .NET proxy classes. In both cases, the Web services are invoked via proxy objects by a single call to a method whose signature matches the Web method. This makes the Message Bridge Web services appear to the message brokers as objects.

As discussed previously, the Message Bridge Web services require an XML representation of the BridgeMessage data class for subject mapping. The .NET Message Broker maps the Label of an MSMQ message to the "Subject" node in the BridgeMessage. If the Extension byte array has data, it is converted into a string and mapped to the "Reply Subject" node. The Body property of the MSMQ message is assigned to the "Data" node.

The Java Message Broker maps the Topic* of the JMS message to the "Subject" node, and the "Data" node is assigned the result of a call to getText() on the JMS message. The "Reply Subject" node is populated with the Topic returned from a call to getJmsReplyTo() on the JMS message object.

*All Java producers assign the Topic to a String property called "SUBJECT" in the JMS message inadvertently through use of a custom library that interacts with a JMS provider. JMS specification states that custom properties can be set in a message. The Java Message Broker then parses the Topic out of this property for routing. It is not possible, when subscribing using a wildcard, for a JMS subscriber to know that Topic the message was published on without this workaround.

Interoperability Is Not Perfect

.NET and Java transport interoperability in the Message Bridge is achieved through integration between the MSMQ Message Broker and Java Message Service, and integration between the JMS Message Broker and .NET Message Service. .NET and Axis both have different implementations of SOAP, so it would be impossible to create proxy classes if the Web services required complex types in their Web methods. For example, if the Web method in the .NET Message service expected a "BridgeMessage" type instead of a System.String, you could not generate the proxy classes in Java (during Axis proxy creation the following error is thrown: .[axis-wsdl2java] java.io.IOException: Type {http://schemas.xmlsoap.org/wsdl/} BridgeMessage is referenced but not defined...).

In particular, the namespace references are different in the WSDL generated by .NET than that generated by Axis. This is a symptom of an even larger issue whereby different vendors have different interpretations of SOAP when dealing with complex types. If complex types must be used, proxy classes can be thrown away and replaced with custom code that creates the SOAP envelope and invokes the Web methods.

This greatly minimizes the effort in integrating new services and reduces the complexity of modifying existing ones. It also gives an IT department greater flexibility in selecting third-party tools for integration—the API programming language doesn't have to be a deciding factor.

But, because proxy classes are convenient and the Message Bridge Web services are internal, proxy classes are used. As a result, the data exchanged between the Message Bridge Web services must be packaged as a primitive type—hence the string arguments of the Web methods.

The Java Message Broker's XML serialization of the "BridgeMessage" object (using WebMethod's Glue) produces an XML string that contains the namespace reference .xsi:type=.n2:BridgeMessage_TYPE.. that the XML deserializer in the .NET Message Service has trouble interpreting. So, before deserializing, the namespace in the XML string is replaced with NULL and the BridgeMessage object is properly reconstructed.

Conclusion

Java and .NET services can successfully integrate via enterprise messaging. The key is to:

  • Apply a similar message transportation and notification standard for .NET and Java services—Publish/Subscribe.
  • Establish a message structure that can be effectively interpreted by services written in both programming languages—XML-based message structure.
  • Integrate disparate messaging systems (MSMQ and JMS) through an interoperable protocol—HTTP, SOAP (Web services).

Logically, after all core components and standards are successfully put in place, services in a heterogeneous architecture integrate with each other by plugging into a single, "seamless," message bus.

This greatly minimizes the effort in integrating new services and reduces the complexity of modifying existing ones. It also gives an IT department greater flexibility in selecting third-party tools for integration—the API programming language doesn't have to be a deciding factor.



About the Author

Jeremy Thomas

Jeremy Thomas has 5 years of system integration and programming experience both in the US and abroad. He works primarily with C# and Java and focuses on integrating workflow applications into distributed systems. Prior to embarking on his professional career, Jeremy majored in Spanish at the University of Colorado.

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Cisco and Intel have harnessed flash memory technology and truly innovative system software to blast through the boundaries of today's I/O-bound server/storage architectures. See how they are bringing real-time responsiveness to data-intensive applications—for unmatched business advantage. Sponsored by Cisco and Intel® Partnering in Innovation

  • Managing your company's financials is the backbone of your business and is vital to the long-term health and viability of your company. To continue applying the necessary financial rigor to support rapid growth, the accounting department needs the right tools to most efficiently do their job. Read this white paper to understand the 10 essentials of a complete financial management system and how the right solution can help you keep up with the rapidly changing business world.

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds