XML to define byte array format

[sb101]

Not sure if this is possible but I would like to map a byte array as defined by XML. For example, if I had a byte array of 8 bytes and use the XML file below:

<DataStructs>
<Struct>
<Name>NumFiles</Name>
<Type>uint16</Type>
<Count>1</Count>
</Struct>
<Struct>
<Name>FileFlags</Name>
<Type>uint8</Type>
<Count>2</Count>
</Struct>
<Struct>
<Name>TxnCounter</Name>
<Type>uint32</Type>
<Count>1</Count>
</Struct>
</DataStructs>


This would result in:

NumFiles = bytes [0] & [1]
FileFlags[2] = bytes [2] & [3]
TxnCounter = [4], [5], [6] & [7]

This works fine. My main problem is how to access this data at run time. I could use a string to identify the field required as follows:

uint32 txnCounter = XMLDataStructures.GetUint32Field(“TxnCounter”);

The main problem is that this will be a runtime check but I would really like this to be a compile time check i.e. If I could do something like:

uint32 txnCounter = XMLDataStructures.GetUint32Field(STR_TXN_COUNTER);

But I do not know what fields I have until I parse the XML code.

Any suggestions would be appreciated.
Thank You.

[Arjay]

What would you hope to gain by this code?

[sb101]

Hi,

Although the byte array size is a fixed size, the data can be interpreted differently depending on the software version. They would like to make changes to the structure (i.e. XML), without having to change the C# code.

For displaying the format of the structure this is fine but some of the elements defined in the XML are used for calculations therefore I need a way to obtain these values. I could create a function that takes a string and returns the relevant value but, as I said in the op, this will be checked at run-time. I would prefer this to be done at compile time.

Hope this clarifies what I am trying to achieve.
Thanks.

[Arjay]

Okay. Does the schema (i.e. format) of the xml change?

[MadHatter]

you can create a class (lets call it Parser) that is xml serializable that have Name, Type, and Count properties. you can deserialize that xml to an array of these classes. from there it should be pretty trivial to load up the array, then iterate over the the array of Parsers and deserialize it into whatever you need.

[sb101]

Hi,

Arjay - No the schema will not change. It will simply have Name, Type and Count for each element.

MadHatter - Once I deserialize the array won't I still will need to use a string to reference the element I require. i.e.

Parser.GetUint32Value("TxnCounter");

I would ideally like to avoid this if possible and use a compile time check. Something like:

Parser.GetUint32Value(XML_STRUCT_TXN_COUNTER);

I guess I could create a header file after parsing the XML which will have an enum of all the elements within the XML file.

enum {
XML_STRUCT_NUM_FILES,
XML_STRUCT_FILE_FLAGS,
XML_STRUCT_TXN_COUNTER
};

[MadHatter]

no you won't have to parse anything. the framework (xml serializer) will take care of mapping.

I wrote up a quick example (I'm using visual studio 2008 sp1, so you may not be able to use the solution directly, but you should be able to create a new console app, copy the files over, and set the xml file's properties to copy to the build folder).

not really knowing the requirements for your project, I took the liberty of assuming that you have some byte array that contains header information, and that information is parsed from the byte array by some xml schema like the one you posted. I'm assuming you can have multiple sets of those in your xml so that each element read out of the xml maps to a set of 3 xml Struct nodes (I could have assumed that the 3 node set defined how to read from the byte array, and would have been just as easy to do, but I didn't implement it that way.) hopefully what I've done shows what I'm doing enough for you to extrapolate the important parts.

I created a class called Device. its the main object that you work with, and does the dirty work of parsing the array based on the xml. It contains an array of TableInfo objects. TableInfo is a class that represents the schema of your xml and the values extracted from the byte array. It is created by a set of helper classes that pull the info from the xml. they are: DataParser. this class is really a dummy class. it is used to map the root level xml (DataStructs) to an array of Struct's which are represented by the Structure class. All these really do is convert the xml to an object w/ properties that's easy to read and iterate. these two objects (DataParser contains a list of Structures as DataStructs contans an array of Struct's) have their data mapped directly from the xml file. Device then iterates over DataParser's Structs array, and builds an array (or 1 if you only have 3 nodes) of TableInfo's using the Structure information contained in the xml file, to pull data from the byte array.

if you have any questions let me know. Hopefully I'm not too far off base, or provide something useful to look at.

http://sanity-free.org/misc/DataParser.zip

[sb101]

Hi MadHatter. Thank you very much for this. It proved to be very helpful.

My only remaining problem, and I'm not sure if this is possible, is if that I'm using the XML to define parameters in the code. The Name element in the XML is really the parameter name.

So If I have:

<Struct>
<Name>HelloMsg</Name>
<Type>uint8</Type>
<Count>10</Count>
</Struct>
<Struct>
<Name>DateStr</Name>
<Type>uint8</Type>
<Count>10</Count>
</Struct>
.... several more structs with different <Name>'s

I could use something like:


StructList.GetStringItem("HelloMsg");


but this string value will be checked at runtime i.e. if i was to write "HlloMsg" by mistake it wouldn't be picked up by compiler. Therefore I would like an enum (or something similar) that would check the value at compile time i.e.


StructList.GetStringItem(HELLO_MSG); // this would return the value defined by <Name>HelloMsg</Name> which is a 10 byte buffer


I think the only way I can do this is to build a header file (enum of <Name>'s supported) from the XML as a pre-compile event.

Thanks again.

[MadHatter]

you can search the items you import for something that matches the name, then return its value, or you could store the items in a dictionary and map each item to the name element. then it becomes a fairly simple lookup:

if( theDict.ContainsKey("HelloMsg") ) return theDict["HelloMsg"];
else return "";

by default, enums are numeric, so its not possible to use an enum for that, but you can use a class w/ public static (or a class with private static members and public static properties) fields which are string:

public class Values {
public const string HelloMsg = "HelloMsg";
public const string FooMsg = "Foo";
}

//...

StructList.GetString(Values.HelloMsg);

const strings are treated like macro's are in C++ so Value.HelloMsg is replaced w/ the value.

I agree, you'd have to build some sort of supported names from something if you wanted compile time checking.

[sb101]

Thanks MadHatter,

This has been very helpful. I think I will use the XML serialization code to obtain the XML dtructures then, as part of the pre-build, build a class of const string values like you suggested for each of the structure elements.

Thanks again for your help. Very much appreciated.

[Arjay]

Here's another take on the XmlSerialization approach using a dictionary to store the DataStruct contained within the DataStructs class.

I've taken the liberty to change the xml to a more compact format.
<?xml version="1.0" encoding="utf-8"?>
<dataStructs>
<dataStruct name="NumFiles" type="uint16" count="1" />
<dataStruct name="FileFlags" type="uint8" count="2" />
<dataStruct name="TxnCounter" type="uint32" count="1" />
</dataStructs>

The DataStructs class is fairly simple. When creating serializable classes, I usually start by generating an xml schema, running the xsd.exe tool and then modify the generated code to add the required functionality.

In this case a static load method was added, an array operator to access the underlying dictionary objects.
[Serializable()]
[XmlType(AnonymousType=true)]
[XmlRoot( ElementName="dataStructs", Namespace="", IsNullable=false)]
public partial class DataStructs
{
public static DataStructs FromXml( string xmlFile )
{
XmlSerializer serializer = new XmlSerializer( typeof( DataStructs ) );

using( Stream s = File.OpenRead( xmlFile ) )
{
return serializer.Deserialize( s ) as DataStructs;
}
}

public DataStruct this[ string name ]
{
get
{
return _dataStructDictionary.ContainsKey( name ) ? _dataStructDictionary[ name ] : default( DataStruct );
}
}

/// <remarks/>
[XmlElementAttribute( ElementName = "dataStruct" )]
public DataStruct[] Structs
{
get
{
// Only works on .Net 3.0 and higher
// return _dataStructDictionary.Values.ToArray( );

List< DataStruct > list = new List< DataStruct >( );
if( 0 < _dataStructDictionary.Count ) { list.AddRange( _dataStructDictionary.Values ); }

return list.ToArray( );
}
set
{
// NOTE: deserialization brings in values in DataStruct[] array form
// In order to facilitate a lookup by name, we store the values in a dictionary.

if( null == value ) return;
if( null == _dataStructDictionary ) _dataStructDictionary = new Dictionary<string, DataStruct>( );

DataStruct[ ] dataStructs = value as DataStruct[ ];

foreach( DataStruct dataStruct in dataStructs )
{
_dataStructDictionary.Add( dataStruct.Name, dataStruct );
}
}
}

private Dictionary<string, DataStruct> _dataStructDictionary = new Dictionary<string, DataStruct>( );
}

The DataStruct class:
[Serializable()]
[DebuggerStepThroughAttribute()]
[XmlType(AnonymousType=true)]
public partial class DataStruct
{
private string _name = String.Empty;
private string _type = String.Empty;
private int _count = 0;

/// <remarks/>
[XmlAttributeAttribute( AttributeName="name" )]
public string Name { get { return _name; } set { _name = value; } }

/// <remarks/>
[XmlAttributeAttribute( AttributeName = "type" )]
public string Type { get { return _type; } set { _type = value; } }

/// <remarks/>
[XmlAttributeAttribute( AttributeName = "count" )]
public int Count { get { return _count; } set { _count = value; } }
}

Usage:
static void Main( string[ ] args )
{
string xmlFile = Path.Combine( Path.GetDirectoryName( Assembly.GetExecutingAssembly( ).Location ), "DataStructs.xml" );

DataStructs dataStructs = DataStructs.FromXml( xmlFile );

DataStruct dsNumFiles = dataStructs[ Constants.NumFiles ];
DataStruct dsFileFlags = dataStructs[ Constants.FileFlags ];
DataStruct dsTxnCounter = dataStructs[ Constants.TxnCounter ];

Console.WriteLine( "By lookup" );
Console.WriteLine( "{0}\t{1}\t{2}", dsNumFiles.Name, dsNumFiles.Type, dsNumFiles.Count );
Console.WriteLine( "{0}\t{1}\t{2}", dsFileFlags.Name, dsFileFlags.Type, dsFileFlags.Count );
Console.WriteLine( "{0}\t{1}\t{2}", dsTxnCounter.Name, dsTxnCounter.Type, dsTxnCounter.Count );

Console.WriteLine( "\nBy enumeration" );
foreach( DataStruct ds in dataStructs.Structs )
{
Console.WriteLine( "{0}\t{1}\t{2}", ds.Name, ds.Type, ds.Count );
}
}

[sb101]

Hi Arjay,

Thanks very much for this. The Dictionary and "this[string name]" function definitely make using the code much easier.

Also, since the XML files could become large I believe the more compact XML, as you have in the example you gave me, will be better for readability.

Thanks again. Both you and MadHatter have been extremely helpful.