Automated White Box Testing with Microsoft PEX Framework

Introduction

Microsoft's PEX and Moles are two frameworks that can generate test suites with high code coverage. The PEX Framework from Microsoft is an automated White Box Testing tool that can be integrated with Visual Studio IDE seamlessly -- it can simply work as a Visual Studio add-in. This article presents how this tool can be used in your applications and discusses why it is useful.

Pre-Requisites

To work with Microsoft PEX framework, you should have the one of the following installed in your system:

  • Microsoft Visual Studio 2005
  • Microsoft Visual Studio 2008 with SP1
  • Microsoft Visual Studio 2010

Note that if you are using Visual Studio Express Edition, you wouldn't be able to use PEX as an add-in. In this case, you would need to use PEX from the command line.

You can download a copy of Microsoft PEX framework from this link: http://visualstudiogallery.msdn.microsoft.com/en-us/905980ee-039a-4192-bd6c-81e6ec2978c7

Black Box and White Box Testing -- A Quick Look

White Box and Black Box are two of the widely used types of testing software applications. While Black Box testing involves an external view of the test object, White Box testing is a type of testing that is done by the developer of the code. It requires knowledge of how the code works and how it is structured. In Black Box testing you have lesser code coverage and doesn't consider all scenarios. In essence, Black Box testing involves testing the functionality of the application as opposed to testing how the internal program structures work.

White Box testing has more code coverage and helps you find more potential errors in your application code. If used properly, White Box testing can help you get a more stable build for your application. The Wikipedia states: "White-box testing (a.k.a. clear box testing, glass box testing, transparent box testing, or structural testing) is a method of testing software that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing). In white-box testing an internal perspective of the system, as well as programming skills, are required and used to design test cases. The tester chooses inputs to exercise paths through the code and determine the appropriate outputs." Reference: http://en.wikipedia.org/wiki/White-box_testing

Understanding PEX -- where does PEX fit in?

Microsoft PEX framework is a Visual Studio add-in that can be used for testing applications that are targeted at the managed environment of the CLR. Microsoft's PEX framework can help improve the quality of the software with minimal developer effort. It can automatically generate almost all possible code paths in your application's code. Moreover, PEX provides support for parameterized unit testing - an extension of traditional unit testing that helps you reduce the test maintenance costs in your project.

The PEX Framework is a Visual Studio add-in that can provide runtime code analysis with minimal developer effort. With this framework, you can easily understand the input output behaviour in the application's source code, create and save automated tests and even write parameterized unit tests and generate test suites seamlessly.

If PEX is installed in your system, you can run PEX by right-clicking on the method on which you would want PEX to be executed and then selecting "Run Pex" from the context menu. You can execute PEX on any method in your application. Note that the PEX attributes and helpers are all contained inside the Microsoft.Pex.Framework.dll assembly. PEX generally generates a test suite that allows for maximum code coverage.

When working in Unit Testing mode, PEX explores the code for any method that contains the PexMethodAttribute attribute as shown below:

  using Microsoft.Pex.Framework;
  [PexMethod]
  
  public static void Extract(string str, int n) 
  {
     String result = str.Substring(0,n);
  }

But before we delve deep into the discussions, let's take a quick tour of what unit testing is all about. So, what is a unit test anyway? A unit test may be defined as a program that can check an aspect of the implementation that is to be tested. A unit is the smallest testable portion of a particular program that is under test. A parameterized unit test is essentially a method that accepts parameters, executes the code to be tested and states assertions. The MSDN states: "The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect. Each unit is tested separately before integrating them into modules to test the interfaces between modules. Unit testing has proven its value in that a large percentage of defects are identified during its use." Reference: http://msdn.microsoft.com/en-us/library/aa292197(VS.71).aspx

To generate parameterized unit test stubs, you can simply right click on the class or the method on which you intend to generate the stubs in your Visual Studio IDE. Such parameterized methods are generally placed inside a type that acts as a test class for unit testing. Here is an example:

  using Microsoft.VisualStudio.QualityTools.UnitTestFramework;
  
  [TestClass]
  
  public partial class Utilities
  {
  
   [PexMethod]
  
   public static void Extract(string str, int n) 
   {
      String result = str.Substring(0,n);
   }
  
  }

The test class may be in turn be marked with the PexClassAttribute to imply that this is the type under test that contains parameterized unit testing methods. The test class shown earlier may be decorated with this attribute as shown in the code listing that follows:

  using Microsoft.VisualStudio.QualityTools.UnitTestFramework;
  
  [TestClass]
  
  [PexClass(typeof(Utilities))]
  
  public partial class Utilities
  {
  
   [PexMethod]
  
   public static void Extract(string str, int n) 
   {
      String result = str.Substring(0,n);
   }
  
  }

Summary

PEX, an automated white box testing framework from Microsoft research, is a great tool that helps in understanding the behaviour of your application's code, helps in debugging and can create a test suite for you that covers all corner cases automatically. This article provided a head start to understanding and working with Microsoft PEX framework for building applications that are error free and robust. Happy reading!

Suggested Readings

http://research.microsoft.com/en-us/um/redmond/projects/pex/wiki/Reference%20Manual.html
http://channel9.msdn.com/tags/PEX/
http://msdn.microsoft.com/en-us/magazine/ee819140.aspx
http://research.microsoft.com/en-us/projects/pex/gettingstarted.aspx
http://research.microsoft.com/en-us/projects/pex/
http://research.microsoft.com/en-us/projects/pex/downloads.aspx
http://www.softwaretestinghelp.com/white-box-testing/

Related Articles



About the Author

Joydip Kanjilal

Microsoft Most Valuable Professional, Author and Speaker. Featured in "MSDN Featured Developer of the Fortnight (India)" a number of times. Winner of Community Credit Awards at www.community-credit.com several times. Authored numerous books and articles in Microsoft .NET and its related technologies. Authored the following books:-- ASP.NET 4.0 Programming (Mc-Graw Hill Publishing) Entity Framework Tutorial (Packt Publishing) Pro Sync Framework (APRESS) Sams Teach Yourself ASP.NET Ajax in 24 Hours (Sams Publishing) ASP.NET Data Presentation Controls Essentials (Packt Publishing)

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • The mobile revolution and the need for mobile apps have created an unprecedented set of challenges for IT organizations. This eBook discuss these challenges and how organizations can address them by making their mobile app development processes more efficient and more effective.

  • Intelligent N+X Redundancy, Placement Affinities, & Future Proofing in the Virtualized Data Center Virtualization brought about the ability to simplify business continuity management in IT. Workload portability and data replication capabilities mean that physical infrastructure failures no longer need impact application services, and they can rapidly be recovered even in the event of complete site failure. However, Enterprises and Service Providers face new challenges ensuring they have enough compute …

Most Popular Programming Stories

More for Developers

RSS Feeds

Thanks for your registration, follow us on our social networks to keep up-to-date