Consuming an RSS Feed with the .NET Framework

Introduction

Welcome to this installment of the .NET Nuts & Bolts column! This particular column harkens back to the style of some of my early columns where I took a problem faced in my every day project work and shared the solution with the masses. In this case, the problem at hand is how to consume an RSS feed programmatically with .NET framework code. This article will cover the use of the System.Net.WebRequest and System.Xml.XmlDocument to do just that.

Really Simple Syndication

I'll keep the introduction brief. I'm going to assume that if you're reading this that you've at least heard of RSS before otherwise there are probably other things you'd be doing with your time. Really Simple Syndication, or RSS, is a series of formats used to publish works on the Internet in a fairly standardized format. An RSS document, which is often referred to as a feed or channel, contains full or summarized text along with metadata about the work such as author and/or publication date. It is basically a structured XML file that is updated on regular intervals. For example, CodeGuru.com has a number of RSS feeds available such as the one that has the latest stories for the .NET/C# channel. We'll use one of the many CodeGuru.com feeds in our example.

Object Model

We'll start by defining a simple object model that represents an RSS channel and its contained publication. There will be three parts to our basic model. The first part is an object that represents the channel. Another is an object that represents the document contained within the channel. The third and final object is an item representing content within the publication. This is an area where there is plenty of theory, but no real right or wrong answer. It all depends on what you're trying to accomplish.

using System;
using System.Collections.Generic;
using System.Net;
using System.Xml;

public class RssChannel
{
  public string Title { get; set; }
  public System.Uri Link { get; set; }
  public string Description { get; set; }
  public RssDocument Rss { get; set; }

  public RssChannel()
  {
this.Rss = new RssDocument();
  }
}

public class RssDocument
{
  public List<Item> Items { get; set; }

  public RssDocument()
  {
this.Items = new List<Item>();
  }
}

public class Item
{
  public string Title { get; set; }
  public string Link { get; set; }
  public string Description { get; set; }
  public string PubDate { get; set; }
}

System.Net.WebRequest

An RSS feed is not much more than combining HTTP requests and XML parsing. To perform the HTTP parsing we use the System.Net.WebRequest and then use the System.Xml.XmlDocument to parse the resulting output. The example code below adds a static method to the RssChannel class to create a new channel instance and populate it with the content.

public static RssChannel Read(System.Uri url)
{
  WebRequest request = WebRequest.Create(url.AbsoluteUri);

  WebResponse response = request.GetResponse();
  XmlDocument xmlDoc = new XmlDocument();
  try
  {
xmlDoc.Load(response.GetResponseStream());
XmlElement rssElement = xmlDoc["rss"];
if (rssElement == null)
{
  return null;
}

XmlElement channelElement = rssElement["channel"];

if (channelElement != null)
{
  	  // Create the channel and set attributes
  RssChannel rssChannel = new RssChannel();
  rssChannel.Title = channelElement["title"].InnerText;
  rssChannel.Link = new Uri(channelElement["link"].InnerText);
  rssChannel.Description = channelElement["description"].InnerText;

	  // Read the content
  XmlNodeList itemElements = 
channelElement.GetElementsByTagName("item");
  
  foreach (XmlElement itemElement in itemElements)
  {
Item item = new Item()
{
  Title = itemElement["title"].InnerText,
  Link = itemElement["link"].InnerText,
  Description = itemElement["description"].InnerText,
  PubDate = itemElement["pubDate"].InnerText
};

rssChannel.Rss.Items.Add(item);
  }
  return rssChannel;
}
else
{
  return null;
}
  }
  catch
  {
return null;
  }
}



Consuming an RSS Feed with the .NET Framework

Sample Console Application

Below is a sample console application that uses our newly defined RssChannel to retrieve content from the .NET/C# RSS feed on CodeGuru.com.

static void Main(string[] args)
{
  RssChannel channel = Program.Read(new Uri(
  "http://www.codeguru.com/icom_includes/feeds/codeguru/rss-csharp.xml"));

  foreach (Item item in channel.Rss.Items)
  {
Console.WriteLine("Title='{0}', Link='{1}', Desc='{2}', PubDate='{3}'", 
item.Title, item.Link, 
item.Description, item.PubDate);
  }
  Console.ReadLine();
}

Plan for Performance

The reason I was tackling the problem of how to consume an RSS feed from within .NET code was for a particular website/web application I'm working on. There are some industry news sources relevant to the site, which are going to be displayed on the user's dashboard at the time of login. The solution was built out to download and display the content. Worked like a champ. That is it worked right up until we deployed it to the environment where we began to consider performance. The public pages, which are optimized for quick loading, were taking 10 to 15 seconds, which can feel like eternity to an end user. After chasing down a number of different avenues for performance gains (restricted use of ViewState, IP address in connection string, indexes on the database columns, etc.) there weren't any significant gains. I was at a loss until I accidently turned off the RSS feed, at which point the page loaded in under a second. It then dawned on me the real-time load of the RSS feed was causing a site responsiveness issue.

The latency involved in retrieving it was slowing down my site! Several of the toolkits available offer caching and other mechanisms to work around this. In my case, I simply moved the process to pull the content to a background process that runs and updates a database of published content and my application now pulls the content natively to avoid delay.

Toolkits/Helpers

There are a number of toolkits and helpers available on the Internet to help with more complex RSS capabilities and formats as well. An example is the ASP.NET RSS Toolkit that is available on Codeplex. It includes an RssDataSource web control to help make it easy to bind RSS to the user interface. Additionally, it contains support for helping you publish an RSS feed of your own through your website.

Summary

We explored how the System.Net.WebRequest can be used to retrieve RSS content from CodeGuru or other RSS feeds on the Internet. Our simple console application output the contents of the feed to the console, which could just have easily been bound to a user interface. We also explored a tip regarding planning for performance.

Future Columns

The topic of the next column is yet to be determined. If you have something else in particular that you would like to see explained here you could reach me at through http://markstrawmyer.com.





About the Author

Mark Strawmyer

Mark Strawmyer is a Senior Architect of .NET applications for large and mid-size organizations. He specializes in architecture, design and development of Microsoft-based solutions. Mark was honored to be named a Microsoft MVP for application development with C# for the fifth year in a row. You can reach Mark at mark.strawmyer@crowehorwath.com.

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • This paper introduces IBM Java on the IBM PowerLinux 7R2 server and describes IBM's implementation of the Java platform, which includes IBM's Java Virtual Machine and development toolkit.

  • Protecting business operations means shifting the priorities around availability from disaster recovery to business continuity. Enterprises are shifting their focus from recovery from a disaster to preventing the disaster in the first place. With this change in mindset, disaster recovery is no longer the first line of defense; the organizations with a smarter business continuity practice are less impacted when disasters strike. This SmartSelect will provide insight to help guide your enterprise toward better …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds