Sending and Receiving Large Files Using WSE 2.0 (DIME)

XML

XML—Yes, it is an eXtremely Magic Language. It has created an industry-standard way to represent data, express data relationships, and interact with data that has been well accepted in the world. Every computing platform today has some sort of XML support. XML became a universal standard because of the fact that XML+HTTP= SOAP and Web services and because it makes the idea of interoperability very real.

WS-Attachments

WSE supports the Direct Internet Message Encapsulation (DIME) protocol that defines a mechanism for passing attachments in a SOAP message. It is often necessary for a Web service to send a large file of text or binary data, such as an image file, in a SOAP message. SOAP messages, by default, are not necessarily a good mechanism for transporting these large files because they are plain-text XML. To be added to a SOAP message, a file must be serialized into XML, which could be more than double the size of the original file. The DIME protocol solves that problem by defining a mechanism for placing the entire contents of the original file outside the SOAP envelope, eliminating the need to serialize the file into XML.

Baseline WebServics

In the .NET world, XML Web services means simple calling class methods over the Internet using XML and HTTP. This sort of calling standard is called Web services. The baseline Web services have some limitations, such as security, attaching Large files or binary data, and routing the message if protocol is different. So, Microsoft, IBM, and VeriSign found s solution for this problem. In the Microsoft .NET world, the tool for this solution is WSE Web Service Enhancement. By using this tool, we can write a secure Web service, attach a large binary file as part of a SOAP message, and use this tool to make a Web service without HTTP protocol. In this article, I will explain how to use WSE 2.0 to attach a large binary file as a part of a SOAP message in .NET Web services. This is called WS-Attachments. Microsoft gave this name to the mechanism called DIME (Direct Internet Message Encapsulation).

In a baseline Web service, we use a base-64 encoding style to attach a binary file as a SOAP message. This won't work well when we want to attach a large file such as a WAV file or big image file.

There is another commonly used mechanism for packaging multiple pieces of data together besides DIME. That is the Multipurpose Internet Mail Extensions (MIME) specification and, in particular, MIME multipart. MIME multipart has been used for a number of purposes, but its most common use is sending e-mail messages with attachments. It makes perfect sense that the problem that DIME addresses could also be solved by using MIME. In fact, there is an earlier specification called SOAP with Attachments that was created with the help of Microsoft; it uses MIME multipart to solve the attachments problem for SOAP messages. So, why was DIME created in the first place?

First of all, it is important to understand some key differences between MIME multipart and DIME. Like DIME, MIME multipart has a number of "data records" that each have a header and a payload. Instead of using data lengths to indicate where the next header begins, MIME multipart uses a separator string. The separator string is provided at the beginning of a MIME message, and then is inserted between each data record. Code parsing a MIME message must look through the data until it finds the separator string, at which point it knows that it has found the next data record.

The MIME multipart approach is designed to be efficient for the sender of a MIME message into some stream of data. The sender does not need to know how much data they are sending before they send it. They simply stream the data until they reach the end and then they append the separator string.

The problem with this approach is the inefficiencies that may occur on the receiving side. If you are receiving a data record, you have no idea how large the data record is. You can guess at how big to allocate your receiving buffer, but you are invariably going to have to deal with situations where there is more data then you allocated. Now, you have to re-allocate your buffer and still you have no idea whether it will be large enough for the incoming data.

A related problem is the difficulty in simply finding the data record boundaries. For instance, an application reading a compound document may not even be interested in the next three data records in a particular stream. However, with a MIME approach to this problem, they will have to inspect each byte of those three data records. The act of looking at each byte in the data to see whether it is the beginning of the boundary string is an excruciatingly painful task compared to the DIME approach, where you can directly step from record to record based on the fact that DIME includes the lengths of the data in the data record header.

Certainly, the buffer size and boundary searching problems are solvable with a MIME-based solution. MIME is very flexible in how its data record headers can be used, and there are numerous situations where people have added content-length headers to their MIME data records. But now, the question arises: Why are there separator strings if we already know the length of the data record? Also, if you mandate adding a content-length MIME header to the data records, you now have lost the flexibility provided by the sender being able to simply stream the data until its end. It will have to know the complete size of the data record before it can even start to send it.

Of course, needing to know the size of the data before sending it is also a solvable problem to address. Just as DIME has support for chunking, a solution for chunking data could be created within MIME multipart as well that would solve this problem. But think about what the solution we are describing would now look like.

Basically, we have come to the conclusion that a MIME multipart solution should 1) include content lengths of the data records, which means that 2) the separator string delineating data records is no longer needed, and 3) a chunking mechanism would have to be defined to resolve the unknown data size issues. If you imagine trying to resolve all these issues within the MIME multipart framework, it is not much of a stretch to think that you would start to wonder about a simpler solution to these problems. If you throw in the added benefit of the ease of parsing the fixed length portion of a DIME record header and the simplicity of hopping between records in a DIME message, it is not hard to understand the appeal of the DIME approach.

It's important to note that simplicity has an additional benefit that is critical to Web services. Interoperability is a gigantic part of why Web services have become as popular and appealing as they are. The key to successful interoperability is simplicity. Code that is simple to write and straightforward to design can easily be reproduced on multiple platforms. When it comes down to sending a compound document from a Windows platform to a Unix platform, if the format of the compound document is straightforward, without a lot of extra rules and with little room for interpretation, the chances are that the document will be understood on both platforms. A good measure of the simplicity is the number of lines of code required to implement the solution. Consider the logic required to implement a DIME solution compared to the logic of a MIME multipart solution. MIME multipart requires special logic to create separator strings that are unlikely to occur in the data being sent. It requires logic to walk through the data to find the separator strings. It requires header parsing logic that can handle a variable number of headers of variable lengths parsed via string comparisons. The most complex part of creating or parsing a DIME record is probably dealing with the bit-order of the integers being passed in the fixed-length portion of the data record headers. Very little complexity means very few bugs and very few interpretations. The result should be easy interoperability.

Sending and Receiving Large Files Using WSE 2.0 (DIME)

What Is WSE 2.0?

Web Services Enhancements 2.0 for Microsoft .NET (WSE) is a .NET class library for building Web services using the latest Web services protocols, including WS-Security, WS-Secure Conversation, WS-Trust, WS-Policy, WS-Security Policy, WS-Addressing, and WS-Attachments. WSE allows you to add these capabilities at design time using code or at deployment time through the use of a policy file.

Requirements for the following code to work:

  1. VS.NET 2003
  2. IIS 5.0
  3. You have to have downloaded and installed WSE 2.0

You are going to create a ASP.NET Web service project with two Web methods. One is to upload a file to the server; the other is to download a file from the server.

Let's start by creating a new Solution in VS.NET and add a ASP.NET Web service Project to it. After adding or creating an ASP.NET project, right-click the project Name on Solution explorer. You will see a new option called WSE. 2.0 Settings. Select that option.

It will pop up a dialog box; in that, check two check boxes as shown below. The second option is applicable only to ASP.NET projects. These options enable SOAP Extension for this project. By using this, we can intercept the SOAP messages between client and server and vice versa.

Now, we need to add two Web methods to the Service1. ASMX file.

First, add a Download File method. This web method will allow downloading a file from the Web server to the client. The SoapContext provides access to the WS-Attachments, WS-Security, WS-Addressing, and other Web services architecture-specific details associated with a SOAP message.

Get the current context of ResponseSoapContext by using the Current property. Create ab instance of DimeAttachment and add the file to it. Then, load it to the ResponseSoapContext class.

[WebMethod]

public void DownloadFile()
{

SoapContext repcntxt = ResponseSoapContext.Current ;
DimeAttachment dimeattch = new DimeAttachment("image/gif",
                           TypeFormat.MediaType,
                           HttpContext.Current.Server.MapPath
                           ("Pandey\\test.gif"));
repcntxt.Attachments.Add(dimeattch);
}

Next, you will add another Web method called UploadFileToWebServer from client. First, create the file on the server and save it to the directory called Pandey. Use the RequestSoapContext class get the File Size. Create a buffer based on File Size; read data from SoapRequest and write it to the file. Then, close the file Handle.

[WebMethod]
public void UploadFileToServer(string filename)
{
FileStream fs = File.Create(HttpContext.Current.Server.
                            MapPath("Pandey\\"+ filename));
   long filesize = RequestSoapContext.Current.Attachments[0].
                   Stream.Length;
   byte [] buffer = new byte[filesize] ;
RequestSoapContext.Current.Attachments[0].Stream.
                   Read(buffer,0,(int)filesize);
fs.Write(buffer,0,(int)filesize );
fs.Close();
}

Client Code

In this part, you will consume the Web service you have written to upload and download the file. Add a Windows Application project to the existing solution. Right-click the References. Now, we need to our Web service reference to this project, so select Add Web reference. This will pop up one dialog window as shown below.

In the URL dropdown box, you can enter the Web Service URL or you can click the link called Web services on the local machine. This link will display all Web services that are hosted on the local IIS from a list of Web services. You can select your Web service from that; it is going to be a Web reference for your client project to test the above Web service.

Before adding a Web reference to this Windows Application project, you add must a Windows application project to the solution. Right-click the newly added project.

Select the option called WSE 2.0 Settings. This will cause the pop-up dialog box in that dialog box. Check the first check box only. You do not need the second option for this type of project; that option's only applicable to ASP.Net projects. (You did the same settings when you added your Web service project.)

Sending and Receiving Large Files Using WSE 2.0 (DIME)

Consuming an UploadFileToServer Web Method

I have to tell you one important thing here. Because I was writing client code for consuming the above Web service, I used the one VS.NET gives, a Standard Web Service; in our case, it's called Service1. However, this did not work. Then I noticed there was one more Services class added when you add a Web reference to this project; it is Service1Wse. You need to use this class. That is why I made it bold in the following code blocks.

private void button1_Click(object sender, System.EventArgs e)
   {
      if( openFileDialog1.ShowDialog()==DialogResult.OK)
      {
         Cursor=Cursors.WaitCursor;
         localhost.Service1Wse obj = new localhost.Service1Wse();

         int totalfL =0;

         DateTime sTime  = DateTime.Now;
         string f = openFileDialog1.FileName;

         string s =new System.IO.FileInfo(f).Extension;
         DimeAttachment dm = new DimeAttachment(s,TypeFormat.
                                                Unchanged,
                                                openFileDialog1.
                                                FileName);
         obj.RequestSoapContext.Attachments.Add(dm);
         obj.UploadFileToServer(new System.IO.FileInfo(f).Name );
         totalfL += 1;



      }
      Cursor = Cursors.Default;

   }

Consuming a Web Method Download File

private void pictureBox1_Click(object sender, System.EventArgs e)
{
   localhost.Service1Wse obj = new localhost.Service1Wse();
   obj.DownloadFile();
   if(obj.ResponseSoapContext.Attachments.Count==1)
   {
      pictureBox1.Image = new Bitmap(obj.ResponseSoapContext.
                                     Attachments[0].Stream);
   }
}

Now, if you intend to upload larger files, meaning go to a total over the 4 Mb limit (that includes SOAP Message and such), you need to make some changes to the web.config file of the Web Service. You need to set two values here that override the value of the maxRequestLength as well as the WSE Attachment limit. Open the web.config and set as desired.

<httpRuntime> Settings

This configures the ASP.NET HTTP runtime settings. This section can be declared at the machine, site, application, and subdirectory levels.

<httpRuntime useFullyQualifiedRedirectUrl="true|false"
             maxRequestLength="size in kbytes"
             executionTimeout="seconds" />

The <httpRuntime> tag supports three attributes:

Attribute Option Description
useFullyQualifiedRedirectUrl   Indicates whether client-side redirects are fully qualified (which is necessary for some mobile controls). If not, relative redirects are instead sent to the client (the default).
true Specifies that client-side redirects are fully qualified.
false Specifies that client-side redirects are not fully qualified.
executionTimeout   Indicates the maximum number of seconds that a request is allowed to execute before being automatically shut down by ASP.NET.
maxRequestLength   Indicates the maximum file upload size supported by ASP.NET. This limit can be used to prevent denial of service attacks caused by users posting large files to the server.

Syntax Example

The following code example specifies the HTTP runtime parameters for an ASP.NET application.

<configuration>
   <system.web>
      <httpRuntime maxRequestLength="4000"
                   useFullyQualifiedRedirectUrl="true"
                   executionTimeout="45" />
    </system.web>
</configuration>

Requirements

Contained within <system.web>.



About the Author

SeenivasaRagavan Ramadurai

Seenivasaragavan Ramadurai is a .NET consultant, He has been working with .NET technology since pre beta releases. Seenivasa background includes Master's in Computer Science and B.Sc. Mathematics. He has over 9 years of software development experience with Microsoft technologies and has extensive experience developing client-server, distributed, Web services, and component based applications using Visual Studio .NET. Before moving to .NET, Seenivasa has worked on MFC, COM, ATL, and Visual C++ based applications. If you are looking for a consulting help, contact him at skbbaba23@gmail.com

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Learn how cloud-based master data management (MDM) empowers your fast-paced business to get the right data to the right place in real time, so you can remain competitive and agile.

  • Live Event Date: September 10, 2014 @ 11:00 a.m. ET / 8:00 a.m. PT Modern mobile applications connect systems-of-engagement (mobile apps) with systems-of-record (traditional IT) to deliver new and innovative business value. But the lifecycle for development of mobile apps is also new and different. Emerging trends in mobile development call for faster delivery of incremental features, coupled with feedback from the users of the app "in the wild". This loop of continuous delivery and continuous feedback is …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds