Tip: HTTP Conditional Get

When the programmer is implementing a download cache library for his/her application to download files from the HTTP 1.1 server, the programmer usually has to write his/her own implementation of determining whether the file is not modified on the server since the last download, hence no need to re-download the file and just read or load from the previously downloaded file on the local folder.

HTTP 1.1 specification supports a special type of HTTP Get, called HTTP Conditional Get. HTTP Conditional Get does not download the file if the file is not modified since the last download. HTTP Get method changes to a HTTP Conditional GET if the request message includes an If-Modified-Since, If-Unmodified-Since, If-Match, If-None-Match, or If-Range header fields. In this article, I am going to show you how to use the If-Modified-Since and If-None-Match header field in HTTP Get request to turn your HTTP GET into HTTP Conditional GET. I will be using .Net 2.0 HTTP classes to demonstrate this method.

For the first download, it is always a HTTP Get request because we do not have the If-Modified-Since date and If-None-Match message digest. So for the first downloads, we must store the ETag, which is the file's message digest, and the last modified date from the server, to be used in the subsequent HTTP Conditional Get requests.

Note: For the ETag, you have to store the ETag: you cannot calculate the ETag message digest yourself, because you do not know what message digest algorithm which the server is using to calculate the message digest.

Note: For last modified date, you have to store the original string as well, if you are not using .Net HTTP classes but some other HTTP libraries; If you store the same date in another string format, some servers may misinterpret it as a different date, thus a 'different' file which needs to be downloaded.

The HTTP Get code example stores the ETag and last modified date from the HttpWebResponse object returned by HttpWebRequest's GetResponse method. To keep the code snippets simple and easy to understand, I have omitted the proper exception handling code.

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Uri);
request.Method = "GET";
request.Credentials = new NetworkCredential(userName, password);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// store the ETag and last modified date of 
// the file to be used in the http conditional get
string eTag = response.Headers[HttpResponseHeader.ETag];
string ifModifiedSince = response.Headers[HttpResponseHeader.LastModified];

// Download code
//==================
Stream strm = response.GetResponseStream(); // Input stream

//Output stream
FileStream fs = new FileStream(outputFile, FileMode.Create, 
                               FileAccess.Write, FileShare.None);

const int ArrSize = 10000;

Byte[] barr = new Byte[ArrSize];

while(true) 
{
   int Result = strm.Read(barr, 0, ArrSize); 

   if (Result == -1 || Result == 0)
      break;

   fs.Write(barr, 0, Result);
}

fs.Flush();
fs.Close();
strm.Close();
response.Close();
// End of Download code
//=======================

The HTTP Conditional Get code example uses the ETag to set the If-None-Match header field and uses last modified date to set the If-Modified-Since header field before getting the HTTP response. HttpWebReponse class will throw a System.Net.WebException with the StatusCode set to NotModified if the file is not modified; we have to catch this exception and check the StatusCode.

   // Set the IfNoneMatch with the ETag we have just stored
   request.Headers[HttpRequestHeader.IfNoneMatch] = eTag;
   // Set the IfModifiedSince with the last modified date we have just stored
   DateTime dt = DateTime.Parse(ifModifiedSince);
   request.IfModifiedSince = dt;
   request.Credentials = new NetworkCredential(userName, password);
   
   HttpWebResponse response = (HttpWebResponse)request.GetResponse();
   
   // Download code
   //==================
   Stream strm = response.GetResponseStream(); // Input stream

   //Output stream
   FileStream fs = new FileStream(outputFile, FileMode.Create, 
                                  FileAccess.Write, FileShare.None);

   const int ArrSize = 10000;
  
   Byte[] barr = new Byte[ArrSize];

   while(true) 
   {
       int Result = strm.Read(barr, 0, ArrSize); 
  
       if (Result == -1 || Result == 0)
            break;

       fs.Write(barr, 0, Result);
   }

   fs.Flush();
   fs.Close();
   strm.Close();
   response.Close();
   // End of Download code
   //=======================
} 
catch(System.Net.WebException ex)
{
   if (ex.Response != null)
   {
      using (HttpWebResponse response = ex.Response as HttpWebResponse)
      {
         if (response.StatusCode == HttpStatusCode.NotModified)
         {
            MessageBox.Show("File has not been updated since your last request");
            return;
         }
         else
            MessageBox.Show(string.Format("Unexpected status code returned: {0}", response.StatusCode));
      }
   }
}

I have made a C# demo which you can download publicly available files from any HTTP 1.1 servers. The first download is always a download. The subsequent downloads would take place, if the file has changed since the first download.

Note: The HTTP Conditional Get could only take place if the server has provided the ETag and last modified date fields in the first download.

Screenshot of the demo application

First download

Subsequent download



About the Author

Wong Shao Voon

I guess I'll write here what I does in my free time, than to write an accolade of skills which I currently possess. I believe the things I does in my free time, say more about me.

When I am not working, I like to watch Japanese anime. I am also writing some movie script, hoping to see my own movie on the big screen one day.

I like to jog because it makes me feel good, having done something meaningful in the morning before the day starts.

I also writes articles for CodeGuru; I have a few ideas to write about but never get around writing because of hectic schedule.

Downloads

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Live Event Date: May 7, 2014 @ 1:00 p.m. ET / 10:00 a.m. PT This eSeminar will explore three popular games engines and how they empower developers to create exciting, graphically rich, and high-performance games for Android® on Intel® Architecture. Join us for a deep dive as experts describe the features, tools, and common challenges using Marmalade, App Game Kit, and Havok game engines, as well as a discussion of the pros and cons of each engine and how they fit into your development …

  • With JRebel, developers get to see their code changes immediately, fine-tune their code with incremental changes, debug, explore and deploy their code with ease (both locally and remotely), and ultimately spend more time coding instead of waiting for the dreaded application redeploy to finish. Every time a developer tests a code change it takes minutes to build and deploy the application. JRebel keeps the app server running at all times, so testing is instantaneous and interactive.

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds