Tip: HTTP Conditional Get

When the programmer is implementing a download cache library for his/her application to download files from the HTTP 1.1 server, the programmer usually has to write his/her own implementation of determining whether the file is not modified on the server since the last download, hence no need to re-download the file and just read or load from the previously downloaded file on the local folder.

HTTP 1.1 specification supports a special type of HTTP Get, called HTTP
Conditional Get. HTTP Conditional Get does not download the file if the file is
not modified since the last download. HTTP Get method changes to a HTTP
Conditional GET if the request message includes an If-Modified-Since,
If-Unmodified-Since, If-Match, If-None-Match, or If-Range header fields. In this
article, I am going to show you how to use the If-Modified-Since and
If-None-Match header field in HTTP Get request to turn your HTTP GET into HTTP
Conditional GET. I will be using .Net 2.0 HTTP classes to demonstrate this
method.

For the first download, it is always a HTTP Get request because we do not have the If-Modified-Since date and If-None-Match message digest. So for the first downloads, we must store the ETag, which is the file’s message digest, and the last modified date from the server, to be used in the subsequent HTTP Conditional Get requests.

Note: For the ETag, you have to store the ETag: you cannot calculate the ETag message digest yourself, because you do not know what message digest algorithm which the server is using to calculate the message digest.

Note: For last modified date, you have to store the original string as well, if you are not using .Net HTTP classes but some other HTTP libraries; If you store the same date in another string format, some servers may misinterpret it as a different date, thus a ‘different’ file which needs to be downloaded.

The HTTP Get code example stores the ETag and last modified date from the HttpWebResponse object returned by HttpWebRequest’s GetResponse method. To keep the code snippets simple and easy to understand, I have omitted the proper exception handling code.

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Uri);
request.Method = "GET";
request.Credentials = new NetworkCredential(userName, password);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// store the ETag and last modified date of
// the file to be used in the http conditional get
string eTag = response.Headers[HttpResponseHeader.ETag];
string ifModifiedSince = response.Headers[HttpResponseHeader.LastModified];

// Download code
//==================
Stream strm = response.GetResponseStream(); // Input stream

//Output stream
FileStream fs = new FileStream(outputFile, FileMode.Create,
                               FileAccess.Write, FileShare.None);

const int ArrSize = 10000;

Byte[] barr = new Byte[ArrSize];

while(true)
{
   int Result = strm.Read(barr, 0, ArrSize);

   if (Result == -1 || Result == 0)
      break;

   fs.Write(barr, 0, Result);
}

fs.Flush();
fs.Close();
strm.Close();
response.Close();
// End of Download code
//=======================

The HTTP Conditional Get code example uses the ETag to set the If-None-Match header field and uses last modified date to set the If-Modified-Since header field before getting the HTTP response. HttpWebReponse class will throw a System.Net.WebException with the StatusCode set to NotModified if the file is not modified; we have to catch this exception and check the StatusCode.

   // Set the IfNoneMatch with the ETag we have just stored
request.Headers[HttpRequestHeader.IfNoneMatch] = eTag;
// Set the IfModifiedSince with the last modified date we have just stored
DateTime dt = DateTime.Parse(ifModifiedSince);
request.IfModifiedSince = dt;
request.Credentials = new NetworkCredential(userName, password);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// Download code
//==================
Stream strm = response.GetResponseStream(); // Input stream

//Output stream
FileStream fs = new FileStream(outputFile, FileMode.Create,
FileAccess.Write, FileShare.None);

const int ArrSize = 10000;

Byte[] barr = new Byte[ArrSize];

while(true)
{
int Result = strm.Read(barr, 0, ArrSize);

if (Result == -1 || Result == 0)
break;

fs.Write(barr, 0, Result);
}

fs.Flush();
fs.Close();
strm.Close();
response.Close();
// End of Download code
//=======================
}
catch(System.Net.WebException ex)
{
if (ex.Response != null)
{
using (HttpWebResponse response = ex.Response as HttpWebResponse)
{
if (response.StatusCode == HttpStatusCode.NotModified)
{
MessageBox.Show(“File has not been updated since your last request”);
return;
}
else
MessageBox.Show(string.Format(“Unexpected status code returned: {0}”, response.StatusCode));
}
}
}

I have made a C# demo which you can download publicly available files from any HTTP 1.1 servers. The first download is always a download. The subsequent downloads would take place, if the file has changed since the first download.

Note: The HTTP Conditional Get could only take place if the server has provided the ETag and last modified date fields in the first download.

Screenshot of the demo application

First download

Subsequent download

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read