Hotmail Exposed

Environment: .NET Framework

Introduction

Hotmail is probably the most popular e-mailing facility found on the Web today. Millions of people around the world use it for everyday communication with friends and relatives. The main reason for its popularity is the fact that you can use the service absolutely free of charge. Users are able to read their e-mails daily by using a Web interface and there is a client program available, called Outlook Express, which most people will only use at home. Only these two methods are officially available to read mail from your Hotmail account. This is strange because Outlook Express is not the safest client to use for reading e-mails, especially the spam-sensitive Hotmail accounts. Secondly, Web mail is generally considered to be an annoyance, that is only there to offer access to your account from locations such as schools and public Internet access points.

No longer! This document will enable you to build your own client, using a sure and solid way to communicate with Hotmail in the same way as Outlook does. It will be shown how the protocol can be used to your own advantage; it isn’t at all hard to understand, either.

To build a working client, we will need to know about the way Outlook communicates with Hotmail. The protocol that is being used is called Httpmail. Even when you are completely unfamiliar with this protocol, it is not hard to understand. It looks like an XML structure in which some elements that you want to find are placed. The server will respond with the same kind of structure with the queried elements filled with information. I will not start explaining how the protocol looks. If you want to find out, it’s available from SourceForge.

Now that the manner of communication is clear, there are some other things that need clarification. For instance, how do we authenticate ourselves, and how the heck are we going to implement such authentication? Usually, this will mean calculating hashes and keeping passwords in a safe place. At first sight, this could seem like a enormous task to implement. It turned out to be not so difficult at all.

After a review of the SourceForge article, a few things are noticeably interesting. The authentication is done via HTTP headers and is described in an RFC (2617). It’s even called HTTP authentication; there have got to be some classes in the .NET Framework that are able to do that! To parse the response in a safe manner, we will make use of XPath. It should be up to the job and there are classes available for that, too!

Building the Client

To build the client, two components need to be created:

• A proxy that is able to ‘speak’ Httpmail and is able to do HTTP authentication as described in the RFC.
• The actual client that accesses Hotmail via the proxy and uses XPath to parse the responses.

Instead of building a custom proxy, another method could be used. Microsoft ships a component called XmlHttp that is able to make XML-based HTTP requests. The use of this class presented several problems:

1. A request for a limited amount of properties, such as the msgfolderroot, returns all the possible responses for that request. So, a query to obtain only the msgfolderroot would return information about all the mailboxes. This isn’t fatal, of course, but it does show that the class could use some work. The proxy class that is built in this document does not show this problem.
2. The component generates weird-looking HTTP packets. For instance, the query is placed in the HTTP headers; it should be sent as a data section. This will probably result in erroneous server responses, and may be causing Problem 1.
3. Most importantly, the component will not work for some e-mail addresses, while the only configuration that can be made is the username and password. This problem is the biggest show-stopper when using XmlHttp.

Before you read the rest of this document, I assume that you have basic knowledge of the following subjects:

• Httpmail
• XPath
• Basic C#
• Cookies

Please note that the code is fully documented in the source files.

Hotmail Proxy

The proxy will be responsible for making HTTP requests to the Hotmail servers. This will require various things to be implemented:

• Make requests using the PROPFIND method, instead of the usual GET and POST.
• Handle redirection.
• Handle cookies.
• Handle HTTP authentication.
• Send XML data to the remote host, which is basically the same as sending a string.

To send the requests and receive the responses, the class will make use of the HttpWebRequest and HttpWebResponse classes. Cookies will be stored in a CookieContainer. And, most importantly, authentication can be done by adding a NetworkCredential object to the HttpWebRequest. This will fully implement HTTP authentication; how easy is that! All classes reside in the System.Net namespace and are part of the .NET Framework. This means easy-to-use and documented code!

Let’s start with building the class framework.

public class HotmailProxy
{
  private CookieContainer ccContainer;

  public HotmailProxy ()
  {
    ccContainer = new CookieContainer();
  }
}

Okay, not too interesting. Now, let’s get on with building the really important part, the method that actually sends the client’s request to a remote host. It will implement all the proxy’s requirements that are stated above!

The method will need to accommodate the request as a byte array, the destination, and the necessary credentials for authentication (which can be null if cookies are set after logging in):

private string SendRequestTo(byte[] requestBytes,
                             Uri destination,
                             NetworkCredential credential)

The first thing that needs to be done is build the HttpWebRequest object. The object will need to have a specific UserAgent header; otherwise, Hotmail will decline the request (pfff, how conceited). Also, it will have a few other headers initialized.

Implementation of the authentication process is done by adding the NetworkCredential object to the request. That object was quite difficult to build; read about it further on.

{
  HttpWebRequest webRequest =
                 (HttpWebRequest)WebRequest.Create(destination);
  webRequest.Method = "PROPFIND";
  webRequest.Accept = "*/*";
  webRequest.AllowAutoRedirect = false;
  webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0;
                                       Windows NT 5.0;
                                       .NET CLR 1.0.3705)";
  webRequest.CookieContainer = new CookieContainer();
  webRequest.ContentLength = requestBytes.Length;
  webRequest.ContentType = "text/xml";
  webRequest.Credentials = credential;
  webRequest.CookieContainer.Add(ccContainer.GetCookies(
                                 destination));

Now that the request has been built, we will write the request to the requested host and try to get a response:

  try
  {
    Stream reqStream = webRequest.GetRequestStream();
    reqStream.Write(requestBytes,0,requestBytes.Length);
    reqStream.Close();
    HttpWebResponse webResponse =
                    (HttpWebResponse)webRequest.GetResponse();

After verification that a response was received, cookies and redirection can be handled. Otherwise, an exception will be thrown (build your own MailException of some sort for this). Cookies should be compared to the stored cookies by matching names. Redirection is done by checking the HTTP status code, and recursively calling this method.

if (webRequest.HaveResponse)
  {
    // First handle cookies
    foreach(Cookie retCookie in webResponse.Cookies)
    {
    bool cookieFound = false;
    foreach(Cookie
            oldCookie in ccContainer.GetCookies(destination))
    {
      if (retCookie.Name.Equals(oldCookie.Name))
      {
        oldCookie.Value = retCookie.Value;
        cookieFound = true;
      }
    }
      if (!cookieFound)
      ccContainer.Add(retCookie);
    }
  // Next is redirection
  if ((webResponse.StatusCode == HttpStatusCode.Found) ||
  (webResponse.StatusCode == HttpStatusCode.Redirect) ||
  (webResponse.StatusCode == HttpStatusCode.Moved) ||
  (webResponse.StatusCode == HttpStatusCode.MovedPerm..y))
  {
    // Get new location and call recursively
    WebHeaderCollection headers = webResponse.Headers;
    return SendRequestTo(requestBytes,new Uri(headers["location"]),
                         credential);
  }

Now that redirection has been handled and all cookies are set, the response stream can be read to receive the final server’s response. This finishes the method.

    // Read response
      StreamReader stream = new StreamReader(
                            webResponse.GetResponseStream());
      string responseString = stream.ReadToEnd();
      stream.Close();
      return responseString;
    }
    throw new Exception("No response received from host.");
  }
  catch(WebException e)
  {
    throw new Exception("Exception occured while sending
                         request.",e);
  }
}

To complete the class, a public interface will need to be provided; that calls the SendRequestTo method. The request is an XML string, so the method will need to translate that string into a byte array. The following code is pretty basic. Check the input, build the byte array, and send away!

public string SendRequest(string request, Uri destination,
                          NetworkCredential credential)
{
  if(request == null || request.Trim().Length == 0)
    throw new ArgumentNullException("request");
  else if (destination == null)
    throw new ArgumentNullException("destination");
  else
  {
    byte[] xmlBytes = Encoding.ASCII.GetBytes(request);
    return SendRequestTo(xmlBytes,destination, credential);
  }
}

Because authentication isn’t done in all the requests, the following method has been made available to make further requests from a selected host after logging in.

public string SendRequest(string request, Uri destination)
{
  return SendRequest(request,destination,null);
}

Hoera! This completes our proxy. It is now able to do all the things we started out to do! It is able to send requests to Hotmail servers across the world.

Hotmail Client

Now, to implement a class that provides access to Hotmail. The following example will show how to connect to Hotmail using the proxy, get the msgfolderroot, and request some information about mailboxes. The msgfolderroot can be queried for information such as mailbox names, message count, and unread count. This will allow clients to determine how many messages are new, without having to download all the messages (which would be a stupid method to implement such behavior). Using this example and the Web page provided at the top of this document, it will be easy to implement the rest of an e-mail client (it was for me!).

First, let’s begin again with the class framework:

public class HotmailClient
{
  private HotmailProxy hHttp = null;

  public HotmailClient()
  {}
}

Connect()

Now, for the only public method, Connect(). This method will connect to Hotmail using HTTP authentication; it will parse the response to obtain the URL of the msgfolderroot. Next, this URL will be used to determine some mailbox information.

First of all, the proxy class and credentials needed for authentication are built:

public void Connect(string username, string password)
{
  hHttp = new HotmailHttp();
  NetworkCredential credentials = new NetworkCredential(username,
                                                        password,
                                                        null);

Pffew, that was hard; one complete line of code. Easy enough, as you can see! The next job will be to build the XML query for the msgfolderroot. (XmlHttp will return all kinds of nonsense about the inbox, advertisements, and others while, as you can see, only the msgfolderroot is requested.)

  string query = "<?xml version='1.0'?>" +
  "<D:propfind xmlns_D='DAV:' " +
  "xmlns:h='http://schemas.microsoft.com/hotmail/' " +
  "xmlns:hm='urn:schemas:httpmail:'>" +
    "<D:prop>" +
      "<hm:msgfolderroot/>" +
    "</D:prop>" +
  "</D:propfind>";

The query and the credentials can be used to get a response from the Hotmail gateway located at http://services.msn.com/svcs/hotmail/httpmail.asp. This will cause several redirections and cause the HTTP authentication to take place. All can be done with just a single line of code.

  try
  {
    string hotmailEntryPoint = "http://services.msn.com/svcs/
                                       hotmail/httpmail.asp";
    string response = hHttp.SendRequest(query,
                                        new Uri(hotmailEntryPoint),
                                        credentials);
    // Verify response
    if (response == null || response.Trim().Length == 0)
      throw new Exception();

The Hotmail server will respond with the URL of the msgfolderroot. This URL is placed in the XML-based response, so it can be found using Xpath. Another method has been built to do just that.

    // Parse the response, further verifying it.
    Uri folderRootUri = ParseConnectResponse(response);

With the now-obtained URL, information about all the mailboxes on the server can be retrieved.

  // Obtain available folders using folderRootUrl
    RetrieveMailboxes(folderRootUri);
  }
  catch(Exception e)
  {
    // Something went wrong.
    throw new MailException("Exception occurred while connecting
                             to remote host.",e);
  }
}

This completes our first method. As you can see, it calls two other methods that we will now construct. These two contain the interesting parts, parsing the response! But first, an important helper method needs to be built. Without it, all XPath queries fail.

CreateNamespaceManager (XmlNameTable table)

To obtain the URL of the folder root, we will use XPath. The problem lies in the fact that XPath will not return results for nodes that are declared in a namespace. It is able to do so, but information about namespaces need to be stated with the XPath query. This can be done by constructing an XmlNamespaceManager that knows about the namespaces in the Hotmail responses. As these namespaces appear to be really constant, a special method can be build to construct the necessary object.

private XmlNamespaceManager CreateNamespaceManager(XmlNameTable
                                                   table)
{
  XmlNamespaceManager m = new XmlNamespaceManager(table);
  m.AddNamespace("hm","urn:schemas:httpmail:");
  m.AddNamespace("D","DAV:");
  m.AddNamespace("m","urn:schemas:mailheader:");
  m.AddNamespace("c","urn:schemas:contacts:");
  m.AddNamespace("h","http://schemas.microsoft.com/hotmail/");
  return m;
}

ParseConnectResponse (string response)

To parse the response, there are two things that need to be done: place the returned response in an XmlDocument and parse the document using XPath to obtain the URL of the msgfolderroot.

private Uri ParseConnectResponse(string response)
{
  try
  {
    // Load response into XmlDocument
    XmlDocument dom = new XmlDocument();
    dom.LoadXml(response);
    // Query XmlDocument for msgfolderroot node.
    string xpath = "//hm:msgfolderroot";
    XmlNamespaceManager context =
                        CreateNamespaceManager(dom.NameTable);
    XmlNode folderRoot = dom.SelectSingleNode(xpath,context);
    // Verify node
    if (folderRoot == null)
      throw new Exception("Node '" + xpath + "' not found.");
    // Get node text and verify
    string folderRootUrl = folderRoot.InnerText;
    if ((folderRootUrl == null) ||
        (folderRootUrl.Trim().Length == 0))
      throw new Exception("No url found in node '" + xpath + "'.");
    try
    {
      // Return the URI; this may result in a
      // UriFormatException
      return new Uri(folderRootUrl);
    }
    catch
    {
      throw new MailException("Url found in node '" + xpath +
                              "' is invalid:" + folderRootUrl);
    }
  }
  catch(Exception e)
  {
    // Something went wrong.
    throw new Exception("Error occured while parsing connect
                         response.",e);
  }
}

The way this method works is pretty basic now. The correct query to make can be determined using the Web page stated in the introduction. It’s also possible to use a network analyzer for this purpose. Probably you will need to use both. To parse the response, the XmlNamespaceManager is built, and XPath is used.

RetrieveMailboxes (URI folderRootUrl)

The last method in this example will retrieve information about all mailboxes on the server. Basically, it does the same thing as the first two methods combined, so you should already be able to build it yourself. The method is shown here for completeness’s sake. It will build a query and send it to the URL of the msgfolderroot now determined.

private void RetrieveMailboxes(Uri folderRootUrl)
{
  try
  {
    // Build the needed query
    string query = "<?xml version='1.0'?>" +
    "<D:propfind xmlns_D='DAV:' " +
    "xmlns:hm='urn:schemas:httpmail:'>" +
    "<D:prop>" +
      "<D:displayname/>" +
      "<hm:special/>" +
      "<hm:unreadcount/>" +
      "<D:visiblecount/>" +
    "</D:prop>" +
    "</D:propfind>";
    // Get a response from server. No Credentials are used!
    string response = hHttp.SendRequest(query,folderRootUrl);
    // Verify response
    if (response == null || response.Trim().Length == 0)
      throw new ApplicationException("No response received from
                                      host.");
    // Load response into XmlDocument
    XmlDocument dom = new XmlDocument();
    dom.LoadXml(response);
    // Query XmlDocument for all mailboxes using XPath
    string xpath = "//D:response";
    XmlNamespaceManager context =
                        CreateNamespaceManager(dom.NameTable);
    XmlNodeList mailBoxNodes = dom.SelectNodes(xpath,context);
    // Parse each node found
    foreach(XmlNode mailBoxNode in mailBoxNodes)
    {
      try
      {
        // Direct mapping using XPath, should not result in any
        // errors as long as Hotmail keeps the protocol the same.
        string type = mailBoxNode.SelectSingleNode(
                      "descendant::hm:special",context).InnerText;
        string nameUrl = mailBoxNode.SelectSingleNode(
                         "descendant::D:href",context).InnerText;
        int visibleCount = Int32.Parse(
                           mailBoxNode.SelectSingleNode(
                           "descendant::D:visiblecount",
                            context).InnerText);
        int unreadCount =
            Int32.Parse(mailBoxNode.SelectSingleNode("
                  descendant::hm:unreadcount",context).InnerText);

        Console.WriteLine("MailBox found: " + type + "\r\n" +
                          "\turl: " +
                          nameUrl + "\r\n" + "\tVisible: " +
                          visibleCount + "\r\n" + "\tUnread: " +
                          unreadCount + "\r\n");
      }
      catch(Exception e)
      {
        Console.WriteLine("Exception occurred while obtaining
                           mailbox info: " + e.Message);
      }
    }
  }
  catch(Exception e)
  {
    // Something went wrong.
    throw new ApplicationException("Error occurred while retrieving
                                    available mailboxes.",e);
  }
}

Conclusion

As you can see, accessing Hotmail from C# can be as easy as any other type of mail server. The code works without any strange loops or use of classes that function only partly. With this code, it should be easy to build your own e-mail client that is able to access Hotmail with the same type of interface as you would build for IMAP or POP. Happy coding!

About the Author

I am a 23 year old student, currently looking for a project so I can finish my bachelor study of Computer Science. I first came in contact with C Sharp because of my work as a developer for various software development projects done at my college. Recent activities include assisting in the development of a large Web-based learning environment to be used at my college’s site. The Hotmail client was a part of that environment. For that project, I’ve also built a comprehensive Web spider based on XML, XPath, and XSLT.

What I like to do is create things; writing code is a very suitable way to do just that! Currently, my main interest lies in writing reliable code and programming in various languages and with various technologies. I am researching Sun’s Jini technology, am very interested in the field of Artificial Intelligence and will build my own computer brain one day! Or, I may shift my attention to the field of management, but hey; Life is long, Time is short. There is much to do and much to see. I will definitely continue coding, although it might not always be my profession. But right now, you could call me a complete coding junkie.

Bio

Wouter van Vugt
23 years old. (17/11/1979)
4th year Computer Science
Hogeschool Arnhem Nijmegen

Reachable at (this may change soon):
wpvugt1@student.han.nl
woutervu@hotmail.com

Download

Download source code – 5 Kb