Building the Right Environment to Support AI, Machine Learning and Deep Learning
Virtually every major database-based application supports text file importing of one form or another. The problem with utilizing the file-importing feature is that you usually spend more time building an infrastructure to deliver a text file to its desired destination than writing code to format the file. Compounding the problem of delivering the file are things such as security and initiating the file creation. Moreover, how do you share common code when file creation is scattered throughout your organization in Web sites and desktop programs? Wouldn't it be nice to have a centralized place where all files are created? With a centralized location, you could more easily configure security, administer the services, and share code. A file broker may be the answer.
The Broker pattern decouples a service implementation from the consumer of the service. (You can find more information on the Broker pattern in the book Integration Patterns, part of the Microsoft Patterns and Practices collection, by Trowbridge, Roxburgh, Hohpe, Manolescu, and Nadhan.) By implementing a Broker pattern and a framework for building common text files, you can build a centralized service for creating and distributing files anywhere in your organization. By using BizTalk 2004, Web services, and the Broker enterprise pattern, my company, Crowe Chizek, built a service that receives an XML message and invokes functions inside a .NET assembly built to create the desired text file.
Before delving into the details of the file broker, take a look at how it functions. The sequence diagram shown in Figure 1 illustrates what happens when the file broker is invoked.
Figure 1. Sequence of File Broker Invocation
The operations are performed in the following order:
- An application capable of consuming a Web service initiates the file creation.
- The Web service creates an XML document from the SOAP message and passes the document on to a BizTalk orchestration.
- From within the orchestration, a CreateCCFileCommand object is created, and the XML document is passed into the CreateCCFileCommand object.
- Based on the contents of the XML document, the CreateCCFileCommand object instantiates a class derived from the interface ICreateCCFileProduct, serializes the XML document into a CroweCreateCCFile object, and passes the CroweCreateCCFile object to an ICreateCCFileProduct-based class.
- The ICreateCCFileProduct-derived class performs the file creation actions and returns the results of the action in the form of a CroweCreateCCFileResult object.
- Execution continues in the CreateCCFileCommand object. The results are serialized into an XML document and the BizTalk orchestration completes.
- Once the orchestration terminates, the Web service completes and returns a response to the application consuming the Web Service.
All actions are performed synchronously. Thus, no action completes until the succeeding action completes.
Now that you have a conceptual view of the application, you can move on to the details, beginning with the BizTalk orchestration.
BizTalk 2004 Orchestration
You may be wondering why I began with the orchestration when the process begins with a Web service. BizTalk 2004 includes a tool called the BizTalk Web Services Publishing Wizard, which generates a Web service from an existing orchestration. Therefore, once you've built the orchestration, the Web service is essentially done. Figure 2 depicts the orchestration.
Figure 2. BizTalk 2004 Orchestration
One of the more striking features of the orchestration is the request/response port called CreateCCFilePort. This port receives an XML document based on the CroweCreateCCFile schema definition and responds with an XML document based on the CroweCreateCCFileResult schema definition. To build a Web service using the BizTalk Web Services Publishing Wizard, the orchestration must have a request/response port. Also, the orchestration performs very few actions. The heart of the application is in the Crowe.CreateCCFile.dll assembly, and it begins with the CreateCCFileCommand object.
The CreateCCFileCommand object is instantiated by the BizTalk 2004 orchestration. A variable in the orchestration called CreateFileCommand contains a reference to the CreateCCFileCommand object. When the orchestration receives an XML document formatted to follow the CroweCreateCCFile XML Schema Definition, it creates a message based on that schema definition and invokes the Execute command on the CreateFileCommand object passing the CroweCreateCCFile and CroweCreateCCFileResult. The file creation process now begins.
The File Creation Process
First, both XmlDocuments are serialized into an object with the same name as the schema definition. CroweCreateCCFile and CroweCreateCCFileResult were generated by using the XSD.exe utility that ships with Visual Studio 2003. (I will explain shortly why it was important to serialize the XMLDocuments objects.) Control now shifts to the CreateCCFileFactory object.
CreateCCFileFactory implements the Factory object. (A discussion of the Factory object design pattern is beyond the scope of this article. You can find more information about it in Design Patterns Elements of Reusable Object-Oriented Software, by Gamma, Helm, Johnson, and Vlissides.) CreateCCFileFactory has only one duty: build an object based on the ICreateCCFileProduct interface. To fulfill this task, it uses information contained in the CroweCreateCCFile objects (which themselves contain the serialized XML document originally passed into the BizTalk 2004 orchestration) by invoking the BizTalk Web service. The CreateCCFileFactory object instantiates the appropriate class using a .NET tool called reflection.
The following code loads the assembly contained in the Object.AssemblyName of the CroweCreateCCFile object and creates the class from the Object.ObjectInstanceName variable containing a namespace in the loaded assembly:
Assembly assm = Assembly.LoadFrom(assemblyName); ICreateCCFileProduct ccFile; ccFile = null; ccFile = (ICreateCCFileProduct)assm.CreateInstance(objName);
.NET reflection comprises a set of classes for dynamically loading assemblies, dynamically generating code, and reading attribute information contained in a .NET assembly. Reflection classes are contained in the System.Reflection namespace, and the variable objName must contain a full path to the assembly.
At this point, control shifts back to the CreateCCFileCommand object. The command object calls the Execute method on the ICreateCCFileProduct-derived class, and the file creation process is initiated.
The sample class included with this article creates a simple text file, but you can do almost anything within the class, which is derived from the ICreateCCFileProduct interface. My company implemented a file creation process that writes multiple files to SharePoint document libraries. An important detail of such an implementation is security configuration. When the Execute function on the ICreateCCFileProduct-derived object is invoked, the code runs using the service account of the BizTalk service. Therefore, wherever you choose to write your file, you must be sure that the BizTalk service account has permissions to create the file in the desired place. An account in the domain is not necessary as long as an account matching the name and password of the BizTalk service account exists on the target server.
Previously, I promised to explain why we chose to serialize the XML files to the CroweCreateCCFile and CroweCreateCCFileResult objects. The primary reason for serializing to an object is that it frees the developer from manipulating the XML document with one of the various XML libraries. A developer simply accesses the objects the way he or she would normally access a .NET object—no special XML skills are necessary. Another good reason is expansion and validation. If, for instance, you want to overload the Execute function, you can provide alternate Execute functions that utilize other objects.
Is BizTalk the Only Way?
At this point you may be wondering why the BizTalk orchestration was necessary. Couldn't this be done without BizTalk? The answer is yes if you want to configure your Web service to run under a special account and are not concerned about handling exceptions should the file creation process fail. Without BizTalk, it would be difficult to implement a rollback on a file creation process that partially ran before failing. Moreover, BizTalk provides a layer for future expansion of your service and a natural audit trail for troubleshooting problems. From within the BizTalk orchestration, for instance, you can perform further validation or actions by adding BizTalk business rules to your orchestration. In the event of a failure, the original XML message is trapped in the BizTalk engine. Nothing is ever lost, making it easier to follow a trail to the origins of the failure.
By using a centralized BizTalk 2004 file creation broker, you will reap the benefits of code sharing and easier configuration. Most importantly, you will save time and money by leveraging existing .NET development skills.