Tip: HTTP Conditional Get

When the programmer is implementing a download cache library for his/her application to download files from the HTTP 1.1 server, the programmer usually has to write his/her own implementation of determining whether the file is not modified on the server since the last download, hence no need to re-download the file and just read or load from the previously downloaded file on the local folder.

HTTP 1.1 specification supports a special type of HTTP Get, called HTTP Conditional Get. HTTP Conditional Get does not download the file if the file is not modified since the last download. HTTP Get method changes to a HTTP Conditional GET if the request message includes an If-Modified-Since, If-Unmodified-Since, If-Match, If-None-Match, or If-Range header fields. In this article, I am going to show you how to use the If-Modified-Since and If-None-Match header field in HTTP Get request to turn your HTTP GET into HTTP Conditional GET. I will be using .Net 2.0 HTTP classes to demonstrate this method.

For the first download, it is always a HTTP Get request because we do not have the If-Modified-Since date and If-None-Match message digest. So for the first downloads, we must store the ETag, which is the file's message digest, and the last modified date from the server, to be used in the subsequent HTTP Conditional Get requests.

Note: For the ETag, you have to store the ETag: you cannot calculate the ETag message digest yourself, because you do not know what message digest algorithm which the server is using to calculate the message digest.

Note: For last modified date, you have to store the original string as well, if you are not using .Net HTTP classes but some other HTTP libraries; If you store the same date in another string format, some servers may misinterpret it as a different date, thus a 'different' file which needs to be downloaded.

The HTTP Get code example stores the ETag and last modified date from the HttpWebResponse object returned by HttpWebRequest's GetResponse method. To keep the code snippets simple and easy to understand, I have omitted the proper exception handling code.

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Uri);
request.Method = "GET";
request.Credentials = new NetworkCredential(userName, password);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// store the ETag and last modified date of 
// the file to be used in the http conditional get
string eTag = response.Headers[HttpResponseHeader.ETag];
string ifModifiedSince = response.Headers[HttpResponseHeader.LastModified];

// Download code
//==================
Stream strm = response.GetResponseStream(); // Input stream

//Output stream
FileStream fs = new FileStream(outputFile, FileMode.Create, 
                               FileAccess.Write, FileShare.None);

const int ArrSize = 10000;

Byte[] barr = new Byte[ArrSize];

while(true) 
{
   int Result = strm.Read(barr, 0, ArrSize); 

   if (Result == -1 || Result == 0)
      break;

   fs.Write(barr, 0, Result);
}

fs.Flush();
fs.Close();
strm.Close();
response.Close();
// End of Download code
//=======================

The HTTP Conditional Get code example uses the ETag to set the If-None-Match header field and uses last modified date to set the If-Modified-Since header field before getting the HTTP response. HttpWebReponse class will throw a System.Net.WebException with the StatusCode set to NotModified if the file is not modified; we have to catch this exception and check the StatusCode.

   // Set the IfNoneMatch with the ETag we have just stored
   request.Headers[HttpRequestHeader.IfNoneMatch] = eTag;
   // Set the IfModifiedSince with the last modified date we have just stored
   DateTime dt = DateTime.Parse(ifModifiedSince);
   request.IfModifiedSince = dt;
   request.Credentials = new NetworkCredential(userName, password);
   
   HttpWebResponse response = (HttpWebResponse)request.GetResponse();
   
   // Download code
   //==================
   Stream strm = response.GetResponseStream(); // Input stream

   //Output stream
   FileStream fs = new FileStream(outputFile, FileMode.Create, 
                                  FileAccess.Write, FileShare.None);

   const int ArrSize = 10000;
  
   Byte[] barr = new Byte[ArrSize];

   while(true) 
   {
       int Result = strm.Read(barr, 0, ArrSize); 
  
       if (Result == -1 || Result == 0)
            break;

       fs.Write(barr, 0, Result);
   }

   fs.Flush();
   fs.Close();
   strm.Close();
   response.Close();
   // End of Download code
   //=======================
} 
catch(System.Net.WebException ex)
{
   if (ex.Response != null)
   {
      using (HttpWebResponse response = ex.Response as HttpWebResponse)
      {
         if (response.StatusCode == HttpStatusCode.NotModified)
         {
            MessageBox.Show("File has not been updated since your last request");
            return;
         }
         else
            MessageBox.Show(string.Format("Unexpected status code returned: {0}", response.StatusCode));
      }
   }
}

I have made a C# demo which you can download publicly available files from any HTTP 1.1 servers. The first download is always a download. The subsequent downloads would take place, if the file has changed since the first download.

Note: The HTTP Conditional Get could only take place if the server has provided the ETag and last modified date fields in the first download.

Screenshot of the demo application

First download

Subsequent download



About the Author

Wong Shao Voon

I guess I'll write here what I does in my free time, than to write an accolade of skills which I currently possess. I believe the things I does in my free time, say more about me.

When I am not working, I like to watch Japanese anime. I am also writing some movie script, hoping to see my own movie on the big screen one day.

I like to jog because it makes me feel good, having done something meaningful in the morning before the day starts.

I also writes articles for CodeGuru; I have a few ideas to write about but never get around writing because of hectic schedule.

Downloads

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Java developers know that testing code changes can be a huge pain, and waiting for an application to redeploy after a code fix can take an eternity. Wouldn't it be great if you could see your code changes immediately, fine-tune, debug, explore and deploy code without waiting for ages? In this white paper, find out how that's possible with a Java plugin that drastically changes the way you develop, test and run Java applications. Discover the advantages of this plugin, and the changes you can expect to see …

  • "Security" is the number one issue holding business leaders back from the cloud. But does the reality match the perception? Keeping data close to home, on premises, makes business and IT leaders feel inherently more secure. But the truth is, cloud solutions can offer companies real, tangible security advantages. Before you assume that on-site is the only way to keep data safe, it's worth taking a comprehensive approach to evaluating risks. Doing so can lead to big benefits.

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds