Tip: HTTP Conditional Get

When the programmer is implementing a download cache library for his/her application to download files from the HTTP 1.1 server, the programmer usually has to write his/her own implementation of determining whether the file is not modified on the server since the last download, hence no need to re-download the file and just read or load from the previously downloaded file on the local folder.

HTTP 1.1 specification supports a special type of HTTP Get, called HTTP Conditional Get. HTTP Conditional Get does not download the file if the file is not modified since the last download. HTTP Get method changes to a HTTP Conditional GET if the request message includes an If-Modified-Since, If-Unmodified-Since, If-Match, If-None-Match, or If-Range header fields. In this article, I am going to show you how to use the If-Modified-Since and If-None-Match header field in HTTP Get request to turn your HTTP GET into HTTP Conditional GET. I will be using .Net 2.0 HTTP classes to demonstrate this method.

For the first download, it is always a HTTP Get request because we do not have the If-Modified-Since date and If-None-Match message digest. So for the first downloads, we must store the ETag, which is the file's message digest, and the last modified date from the server, to be used in the subsequent HTTP Conditional Get requests.

Note: For the ETag, you have to store the ETag: you cannot calculate the ETag message digest yourself, because you do not know what message digest algorithm which the server is using to calculate the message digest.

Note: For last modified date, you have to store the original string as well, if you are not using .Net HTTP classes but some other HTTP libraries; If you store the same date in another string format, some servers may misinterpret it as a different date, thus a 'different' file which needs to be downloaded.

The HTTP Get code example stores the ETag and last modified date from the HttpWebResponse object returned by HttpWebRequest's GetResponse method. To keep the code snippets simple and easy to understand, I have omitted the proper exception handling code.

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Uri);
request.Method = "GET";
request.Credentials = new NetworkCredential(userName, password);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// store the ETag and last modified date of 
// the file to be used in the http conditional get
string eTag = response.Headers[HttpResponseHeader.ETag];
string ifModifiedSince = response.Headers[HttpResponseHeader.LastModified];

// Download code
//==================
Stream strm = response.GetResponseStream(); // Input stream

//Output stream
FileStream fs = new FileStream(outputFile, FileMode.Create, 
                               FileAccess.Write, FileShare.None);

const int ArrSize = 10000;

Byte[] barr = new Byte[ArrSize];

while(true) 
{
   int Result = strm.Read(barr, 0, ArrSize); 

   if (Result == -1 || Result == 0)
      break;

   fs.Write(barr, 0, Result);
}

fs.Flush();
fs.Close();
strm.Close();
response.Close();
// End of Download code
//=======================

The HTTP Conditional Get code example uses the ETag to set the If-None-Match header field and uses last modified date to set the If-Modified-Since header field before getting the HTTP response. HttpWebReponse class will throw a System.Net.WebException with the StatusCode set to NotModified if the file is not modified; we have to catch this exception and check the StatusCode.

   // Set the IfNoneMatch with the ETag we have just stored
   request.Headers[HttpRequestHeader.IfNoneMatch] = eTag;
   // Set the IfModifiedSince with the last modified date we have just stored
   DateTime dt = DateTime.Parse(ifModifiedSince);
   request.IfModifiedSince = dt;
   request.Credentials = new NetworkCredential(userName, password);
   
   HttpWebResponse response = (HttpWebResponse)request.GetResponse();
   
   // Download code
   //==================
   Stream strm = response.GetResponseStream(); // Input stream

   //Output stream
   FileStream fs = new FileStream(outputFile, FileMode.Create, 
                                  FileAccess.Write, FileShare.None);

   const int ArrSize = 10000;
  
   Byte[] barr = new Byte[ArrSize];

   while(true) 
   {
       int Result = strm.Read(barr, 0, ArrSize); 
  
       if (Result == -1 || Result == 0)
            break;

       fs.Write(barr, 0, Result);
   }

   fs.Flush();
   fs.Close();
   strm.Close();
   response.Close();
   // End of Download code
   //=======================
} 
catch(System.Net.WebException ex)
{
   if (ex.Response != null)
   {
      using (HttpWebResponse response = ex.Response as HttpWebResponse)
      {
         if (response.StatusCode == HttpStatusCode.NotModified)
         {
            MessageBox.Show("File has not been updated since your last request");
            return;
         }
         else
            MessageBox.Show(string.Format("Unexpected status code returned: {0}", response.StatusCode));
      }
   }
}

I have made a C# demo which you can download publicly available files from any HTTP 1.1 servers. The first download is always a download. The subsequent downloads would take place, if the file has changed since the first download.

Note: The HTTP Conditional Get could only take place if the server has provided the ETag and last modified date fields in the first download.

Screenshot of the demo application

First download

Subsequent download



About the Author

Wong Shao Voon

I guess I'll write here what I does in my free time, than to write an accolade of skills which I currently possess. I believe the things I does in my free time, say more about me.

When I am not working, I like to watch Japanese anime. I am also writing some movie script, hoping to see my own movie on the big screen one day.

I like to jog because it makes me feel good, having done something meaningful in the morning before the day starts.

I also writes articles for CodeGuru; I have a few ideas to write about but never get around writing because of hectic schedule.

Downloads

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • On-demand Event Event Date: September 10, 2014 Modern mobile applications connect systems-of-engagement (mobile apps) with systems-of-record (traditional IT) to deliver new and innovative business value. But the lifecycle for development of mobile apps is also new and different. Emerging trends in mobile development call for faster delivery of incremental features, coupled with feedback from the users of the app "in the wild." This loop of continuous delivery and continuous feedback is how the best mobile …

  • A modern mobile IT strategy is no longer an option, it is an absolute business necessity. Today's most productive employees are not tied to a desk, an office, or a location. They are mobile. And your company's IT strategy has to be ready to support them with easy, reliable, 24/7 access to the business information they need, from anywhere in the world, across a broad range of communication devices. Here's how some of the nation's most progressive corporations are meeting the many needs of their mobile workers …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds