Consuming an RSS Feed with the .NET Framework

Introduction

Welcome to this installment of the .NET Nuts & Bolts column! This particular column harkens back to the style of some of my early columns where I took a problem faced in my every day project work and shared the solution with the masses. In this case, the problem at hand is how to consume an RSS feed programmatically with .NET framework code. This article will cover the use of the System.Net.WebRequest and System.Xml.XmlDocument to do just that.

Really Simple Syndication

I'll keep the introduction brief. I'm going to assume that if you're reading this that you've at least heard of RSS before otherwise there are probably other things you'd be doing with your time. Really Simple Syndication, or RSS, is a series of formats used to publish works on the Internet in a fairly standardized format. An RSS document, which is often referred to as a feed or channel, contains full or summarized text along with metadata about the work such as author and/or publication date. It is basically a structured XML file that is updated on regular intervals. For example, CodeGuru.com has a number of RSS feeds available such as the one that has the latest stories for the .NET/C# channel. We'll use one of the many CodeGuru.com feeds in our example.

Object Model

We'll start by defining a simple object model that represents an RSS channel and its contained publication. There will be three parts to our basic model. The first part is an object that represents the channel. Another is an object that represents the document contained within the channel. The third and final object is an item representing content within the publication. This is an area where there is plenty of theory, but no real right or wrong answer. It all depends on what you're trying to accomplish.

using System;
using System.Collections.Generic;
using System.Net;
using System.Xml;

public class RssChannel
{
  public string Title { get; set; }
  public System.Uri Link { get; set; }
  public string Description { get; set; }
  public RssDocument Rss { get; set; }

  public RssChannel()
  {
this.Rss = new RssDocument();
  }
}

public class RssDocument
{
  public List<Item> Items { get; set; }

  public RssDocument()
  {
this.Items = new List<Item>();
  }
}

public class Item
{
  public string Title { get; set; }
  public string Link { get; set; }
  public string Description { get; set; }
  public string PubDate { get; set; }
}

System.Net.WebRequest

An RSS feed is not much more than combining HTTP requests and XML parsing. To perform the HTTP parsing we use the System.Net.WebRequest and then use the System.Xml.XmlDocument to parse the resulting output. The example code below adds a static method to the RssChannel class to create a new channel instance and populate it with the content.

public static RssChannel Read(System.Uri url)
{
  WebRequest request = WebRequest.Create(url.AbsoluteUri);

  WebResponse response = request.GetResponse();
  XmlDocument xmlDoc = new XmlDocument();
  try
  {
xmlDoc.Load(response.GetResponseStream());
XmlElement rssElement = xmlDoc["rss"];
if (rssElement == null)
{
  return null;
}

XmlElement channelElement = rssElement["channel"];

if (channelElement != null)
{
  	  // Create the channel and set attributes
  RssChannel rssChannel = new RssChannel();
  rssChannel.Title = channelElement["title"].InnerText;
  rssChannel.Link = new Uri(channelElement["link"].InnerText);
  rssChannel.Description = channelElement["description"].InnerText;

	  // Read the content
  XmlNodeList itemElements = 
channelElement.GetElementsByTagName("item");
  
  foreach (XmlElement itemElement in itemElements)
  {
Item item = new Item()
{
  Title = itemElement["title"].InnerText,
  Link = itemElement["link"].InnerText,
  Description = itemElement["description"].InnerText,
  PubDate = itemElement["pubDate"].InnerText
};

rssChannel.Rss.Items.Add(item);
  }
  return rssChannel;
}
else
{
  return null;
}
  }
  catch
  {
return null;
  }
}



Consuming an RSS Feed with the .NET Framework

Sample Console Application

Below is a sample console application that uses our newly defined RssChannel to retrieve content from the .NET/C# RSS feed on CodeGuru.com.

static void Main(string[] args)
{
  RssChannel channel = Program.Read(new Uri(
  "http://www.codeguru.com/icom_includes/feeds/codeguru/rss-csharp.xml"));

  foreach (Item item in channel.Rss.Items)
  {
Console.WriteLine("Title='{0}', Link='{1}', Desc='{2}', PubDate='{3}'", 
item.Title, item.Link, 
item.Description, item.PubDate);
  }
  Console.ReadLine();
}

Plan for Performance

The reason I was tackling the problem of how to consume an RSS feed from within .NET code was for a particular website/web application I'm working on. There are some industry news sources relevant to the site, which are going to be displayed on the user's dashboard at the time of login. The solution was built out to download and display the content. Worked like a champ. That is it worked right up until we deployed it to the environment where we began to consider performance. The public pages, which are optimized for quick loading, were taking 10 to 15 seconds, which can feel like eternity to an end user. After chasing down a number of different avenues for performance gains (restricted use of ViewState, IP address in connection string, indexes on the database columns, etc.) there weren't any significant gains. I was at a loss until I accidently turned off the RSS feed, at which point the page loaded in under a second. It then dawned on me the real-time load of the RSS feed was causing a site responsiveness issue.

The latency involved in retrieving it was slowing down my site! Several of the toolkits available offer caching and other mechanisms to work around this. In my case, I simply moved the process to pull the content to a background process that runs and updates a database of published content and my application now pulls the content natively to avoid delay.

Toolkits/Helpers

There are a number of toolkits and helpers available on the Internet to help with more complex RSS capabilities and formats as well. An example is the ASP.NET RSS Toolkit that is available on Codeplex. It includes an RssDataSource web control to help make it easy to bind RSS to the user interface. Additionally, it contains support for helping you publish an RSS feed of your own through your website.

Summary

We explored how the System.Net.WebRequest can be used to retrieve RSS content from CodeGuru or other RSS feeds on the Internet. Our simple console application output the contents of the feed to the console, which could just have easily been bound to a user interface. We also explored a tip regarding planning for performance.

Future Columns

The topic of the next column is yet to be determined. If you have something else in particular that you would like to see explained here you could reach me at through http://markstrawmyer.com.





About the Author

Mark Strawmyer

Mark Strawmyer is a Senior Architect of .NET applications for large and mid-size organizations. He specializes in architecture, design and development of Microsoft-based solutions. Mark was honored to be named a Microsoft MVP for application development with C# for the fifth year in a row. You can reach Mark at mark.strawmyer@crowehorwath.com.

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Learn How A Global Entertainment Company Saw a 448% ROI Every business today uses software to manage systems, deliver products, and empower employees to do their jobs. But software inevitably breaks, and when it does, businesses lose money -- in the form of dissatisfied customers, missed SLAs or lost productivity. PagerDuty, an operations performance platform, solves this problem by helping operations engineers and developers more effectively manage and resolve incidents across a company's global operations. …

  • Live Event Date: December 18, 2014 @ 2:00 p.m. ET / 11:00 a.m. PT The Internet of Things (IoT) incorporates physical devices into business processes using predictive analytics. While it relies heavily on existing Internet technologies, it differs by including physical devices, specialized protocols, physical analytics, and a unique partner network. To capture the real business value of IoT, the industry must move beyond customized projects to general patterns and platforms. Check out this upcoming webcast …

Most Popular Programming Stories

More for Developers

RSS Feeds