.NET Azure Queuing and Serialization

Serialization turns data structures into bytes for either transportation or persistence.  Understanding Serialization is important if a developer is contemplating Azure .NET Queuing on either Azure Service Bus or Azure Storage Queues.  While the Azure .NET libraries handle transporting bytes and controlling message consumption; message data structures are left to the developer.  Serialization is the quickest way to turn a data structure into bytes.  A short path to doing Serialization is to leverage the .NET Data Contract Serialization.  What follows are common patterns to effectively apply Data Contract Serialization.

Data Contract Basics

Both the CloudQueueMessage class from Azure Storage Queues and the BrokeredMessage class from Azure Service Bus include support to create messages from a Data Contract or a byte array.   Data Contract components are found in the .NET System.Runtime.Serialization reference.  The following code demonstrates a basic Data Contract.

[DataContract]
class TestContract1
{
    public TestContract1()
    {
        this.Value1 = "ThisIsValue1";
        this.ValueInt = 100;
        this.ValueTimeSpan = new TimeSpan(0, 10, 0);
        this.ValueEnum = TestEnum.Enum2;
 
        this.ValueStringBuilder = new StringBuilder();
 
        this.ValueStringBuilder.Append("StringBuilderText");
    }
 
    public string Value1Missing { get; set; }
 
    [DataMember]
    public string Value1 { get; private set; }
    [DataMember]
    public int ValueInt { get; set; }
    [DataMember]
    public TimeSpan ValueTimeSpan { get; set; }
    [DataMember]
    public TestEnum ValueEnum { get; set; }
    [DataMember]
    public StringBuilder ValueStringBuilder { get; set; }
}
 

Data Contracts are attribute driven.  The DataContractAttribute tags a Data Contract class implementation.  DataMemberAttribute ropes a property into Serialization.  Omitting DataMemberAttribute, like in the Value1Missing example property, excludes the property from Serialization.  Serialized properties need not be public, but all serialized properties must have a Get and Set.  The following XML is generated from the Data Contract sample code.

<TestContract1 xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Test.DataContract">
<Value1>ThisIsValue1</Value1>
<ValueEnum>Enum2</ValueEnum>
<ValueInt>100</ValueInt>
<ValueStringBuilder xmlns:d2p1="http://schemas.datacontract.org/2004/07/System.Text">
<m_MaxCapacity xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int" xmlns="">2147483647</m_MaxCapacity>
<Capacity xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int" xmlns="">32</Capacity>
<m_StringValue xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:string" xmlns="">StringBuilderText</m_StringValue>
<m_currentThread xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int" xmlns="">0</m_currentThread>
</ValueStringBuilder>
<ValueTimeSpan>PT10M</ValueTimeSpan>
</TestContract1>

Data Contracts can be nested.  In the sample; the StringBuilder class is nested within the TestContract class.  Most .NET primitives are supported and any reference class marked Serializable is Serializable in a Data Contract.  Serialized classes often include serialized properties of their own.  Aside from cluttering the XML up; the additional properties add to the Serialized size. 

The example includes Enum values.  Though enums serialize easily, a developer should consider how an Enum will be consumed across the distributed components of a system.  For example: will the Enum change and will the assembly containing the Enum be on every machine consuming the message.  I’ve always preferred to serialize an Enum into a String property and opt to invoke the Enum.Parse to convert the string to an Enum.  More on how to work with Enum values can be found in the MSDN documentation here http://msdn.microsoft.com/en-us/library/aa347875(v=vs.100).aspx .

Once a Data Contract is created; Serialization depends on the DataContractSerializer class to turn the class into XML.

Serialization and Deserialization

There are multiple ways to employ the DataContractSerializer.  Following is a straightforward way to turn a Data Contract into a String.

static StringBuilder SerializeToStringBuilder<T>(T obj)
{
    var sb = new StringBuilder();
    var xmlWrite = new XmlTextWriter(new StringWriter(sb));
    DataContractSerializer serializer = new DataContractSerializer(typeof(T));
    serializer.WriteObject(xmlWrite, obj);
    return sb;
}
 

The code writes the resulting XML to a StringBuilder class.  DataContractSerializer is a generic.  The generic enforces type safety on the process.  Similar techniques exist to write a Data Contract to a byte array via a MemoryStream.

Deserialization works in the opposite direction turning XML into a class instance.  The following code performs Deserialization.

static T DeSerializeToClass<T>(string xmlData)
{
    T obj;
    DataContractSerializer serializer = new DataContractSerializer(typeof(T));            
    var reader = new XmlTextReader(new StringReader(xmlData));
    obj = (T)serializer.ReadObject(reader);
    return obj;
}
 

An interesting aspect to Deserialization is that as long as the Data Contract name and namespace match the name and namespace on the class being instantiated; it’s not necessary to instantiate the original serialized class.  The following Data Contract demonstrates how to set the namespace and name.

[DataContract(Namespace = "http://data.mydomain.com/2012/10", Name = "Test.DataContract1")]
class TestContract1
{
    public TestContract1()
    {
        this.Value1 = "ThisIsValue1";
        this.ValueInt = 100;
        this.ValueTimeSpan = new TimeSpan(0, 10, 0);
        this.ValueEnum = TestEnum.Enum2;
 
        this.ValueStringBuilder = new StringBuilder();
 
        this.ValueStringBuilder.Append("StringBuilderText");
    }
 
    public string Value1Missing { get; set; }
 
    [DataMember]
    public string Value1 { get; private set; }
    [DataMember]
    public int ValueInt { get; set; }
    [DataMember]
    public TimeSpan ValueTimeSpan { get; set; }
    [DataMember]
    public TestEnum ValueEnum { get; set; }
    [DataMember]
    public StringBuilder ValueStringBuilder { get; set; }
}

Following is the serialized class with the new namespace and name.

<Test.DataContract1 xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://data.mydomain.com/2012/10">
<Value1>ThisIsValue1</Value1>
<ValueEnum>Enum2</ValueEnum>
<ValueInt>100</ValueInt>
<ValueStringBuilder xmlns:d2p1="http://schemas.datacontract.org/2004/07/System.Text">
  <m_MaxCapacity xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int" xmlns="">2147483647</m_MaxCapacity>
  <Capacity xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int" xmlns="">32</Capacity>
  <m_StringValue xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:string" xmlns="">StringBuilderText</m_StringValue>
  <m_currentThread xmlns:d3p1="http://www.w3.org/2001/XMLSchema" i:type="d3p1:int" xmlns="">0</m_currentThread>
</ValueStringBuilder>
<ValueTimeSpan>PT10M</ValueTimeSpan>
</Test.DataContract1>

Serializing and Deserializing to different classes can make multi version co-existence easier.

For message routing and other operational reasons it may be useful to create a common message format and embed the true payload within the message.

Envelopes

Messaging solutions can often employ multiple message payloads.  It’s often helpful to encapsulate a message payload within another more generic message.  This technique is often referred to as following an Envelope Pattern.   The Envelope Pattern is especially useful if multiple message types arrive in on the same Queue.  One approach to handling multiple formats is to add an Object class property and assign the payload to the Object.  The following code demonstrates this approach.

[DataContract(Namespace = "http://data.mydomain.com/2012/10", Name = "Test.TestContractCanonical")]
class TestContractCanonical1
{
    public TestContractCanonical1(int messageType)
    {
        this.MessageType = messageType;
        this.MessageId = Guid.NewGuid().ToString();
    }
 
    [DataMember]
    public string MessageId { get; private set; }
 
    [DataMember]
    public int MessageType { get; set; }
 
    [DataMember]
    public object Payload { get; set; }
 
}
 

As long as the assigned property is another Data Contract, Serialization works fine; but requires some additional DataContractSerializer parameters.  Following is the Serialization and Deserialization code with parameter support for the additional class.

static T DeSerializeToClass<T>(string xmlData, List<Type> internalTypes)
{
    T obj;
    DataContractSerializer serializer = new DataContractSerializer(typeof(T), internalTypes);            
    var reader = new XmlTextReader(new StringReader(xmlData));
    obj = (T)serializer.ReadObject(reader);
    return obj;
}
 
static StringBuilder SerializeToStringBuilder<T>(T obj, List<Type> internalTypes)
{
    var sb = new StringBuilder();
    var xmlWrite = new XmlTextWriter(new StringWriter(sb));
    DataContractSerializer serializer = new DataContractSerializer(typeof(T), internalTypes);
    serializer.WriteObject(xmlWrite, obj);
    return sb;
}
 

The Known types parameter must include types for additional classes beyond what is supported in the .NET Framework.  One other twist to the Envelope Patter; Payload property approach is to assign the property a serialized string.

Serialized Payload

A serialized payload can be helpful in a number of situations.  Foremost are situations where the Known types are not known until the message is unpacked and inspected.  A developer can often query the raw XML with, for example, XPath, but it’s often easier to work with .NET code.  The following code demonstrates the serialized payload approach.

static void CommonWithSerialized()
{
    var resolveList = new List<Type>();//This can be empty.
 
    var c = new TestContractCanonical2(100);
 
    c.Payload = SerializeToStringBuilder<TestContract2>(new TestContract2(), resolveList).ToString();
 
    var str = SerializeToStringBuilder<TestContractCanonical2>(c, resolveList).ToString();
 
    var c2 = DeSerializeToClass<TestContractCanonical2>(str, resolveList);
 
    if (c2.MessageType == 100)
    {
        var cSub = DeSerializeToClass<TestContract2>(c2.Payload, resolveList);
 
    }
}
 

Strings are reference classes; so once the class is deserialized the payload can, for example, be passed to methods without consuming an inordinate amount of resources.  If the serialized class isn’t visible to the current assembly because it’s, for example, internal in another assembly; serialization can be deferred to the assembly containing the class.  This can be a useful way to hide an internal class implementation.

Conclusion

Understanding Serialization is important if a developer is contemplating Azure .NET Queuing on either Azure Service Bus or Azure Storage Queues.  Data Contract Serialization is an easy way to achieve Serialization.

Resources

"Using Data Contracts"

http://msdn.microsoft.com/en-us/library/ms733127(v=vs.100).aspx

"Envelope Wrapper Pattern"

http://www.enterpriseintegrationpatterns.com/EnvelopeWrapper.html



Related Articles

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Live Event Date: December 11, 2014 @ 1:00 p.m. ET / 10:00 a.m. PT Market pressures to move more quickly and develop innovative applications are forcing organizations to rethink how they develop and release applications. The combination of public clouds and physical back-end infrastructures are a means to get applications out faster. However, these hybrid solutions complicate DevOps adoption, with application delivery pipelines that span across complex hybrid cloud and non-cloud environments. Check out this …

  • With the average hard drive now averaging one terabyte in size, the fallout from the explosion of user-created data has become an overwhelming volume of potential evidence that law-enforcement and corporate investigators spend countless hours examining. Join Us and SANS' Rob Lee for our 45-minute webinar, A Triage and Collection Strategy for Time-Sensitive Investigations, will demonstrate how to: Identify the folders and files that often contain key insights Reduce the time spent sifting through content by …

Most Popular Programming Stories

More for Developers

RSS Feeds