Application Security Testing: An Integral Part of DevOps
Many of you have read or heard me talk about NUnit, the open source testing tool from www.nunit.org. And, long-time readers of this column or my books will know that I have written about project templates in .NET before. In this two-part article, I would like to combine both technologies and demonstrate a practical application for project templates that incorporate a NUnit project with TraceListeners yielding a new project template, the Test Project.
If you have read any of the aforementioned articles, rest assured that you may both use those articles for cross-reference, and I have included some completely new material herein. If you are very comfortable with any one or more sections of either parts of this two-part article, feel free to skip that section.
Creating an Example Project
I test everything I write. I used to refer to this testing as scaffolding. Scaffolding is literally where extra test code is added to my product code. The scaffold is comprised of methods that create objects and invoke behaviors, looking for a predetermined result. A few other authors used the term scaffolding before me but it never really caught on.
NUnit is like scaffolding. NUnit uses Reflection in .NET to load and create objects and methods attributed with custom attributes defined by the NUnit framework. NUnit then can load the class library and invoke operations on these attributed classes and methods. The methods themselves can be simple or complex and just about anything. An enhancement I like—and maybe one that should be added to NUnit, which is open source after all—is a custom TraceListener.
A custom TraceListener permits me to send debug statements—Trace.WriteLine—from my product-code to the NUnit console window. Consequently, I have the option of relying on the very simple Red-Green Fail-Pass color coding prescribed by NUnit or a more detailed examination of test results in NUnit's Console.Out window.
Listing 1 demonstrates a stubbed NUnit test project containing all of these elements. To recap: The listing is part of a Class Library that includes a reference to the NUnit.Framework.dll from NUnit version 2.1, a custom TraceListener, and some basic setup and teardown scaffolding. This prototype code will become our project template in Part 2.
Listing 1: An NUnit stub class library with a custom TraceListener.
Imports NUnit.Framework Imports System.Diagnostics Friend Class MyListener Inherits TraceListener Public Overloads Overrides Sub Write(ByVal message As String) Console.Write(message) End Sub Public Overloads Overrides Sub WriteLine(ByVal message As String) Console.WriteLine(message) End Sub End Class <TestFixture()> _ Public Class Test Private Shared listener As MyListener = New MyListener <SetUp()> _ Public Sub Init() If (Not Trace.Listeners.Contains(listener)) Then Trace.Listeners.Add(listener) End If End Sub <TearDown()> _ Public Sub Deinit() If (Trace.Listeners.Contains(listener)) Then Trace.Listeners.Remove(listener) End If End Sub <Test()> _ Public Sub Test1() Assertion.Assert("Add a boolean test here", False) End Sub <Test()> _ Public Sub Test2() Assertion.Fail("Intentionally fail if this code is run") End Sub <Ignore("Not ready yet!")> _ Public Sub Test3() ' Change the attribute from Ignore to Test ' when this test is finished End Sub End Class
After creating the Class Library, the next step was to add a reference to NUnit.Framework.dll. Of course, you will need to go to nunit.org and download the latest version of NUnit before you can reference that assembly. The next step is to add an Imports statement for the NUnit.Framework and System.Diagnostics namespaces. Next, I created an internal class called MyListener; MyListener implements the abstract base class TraceListener via inheritance. Finally, the test class is stubbed out.
You can add more than one test class, but for NUnit to find your test classes you need to attribute each with the TestFixtureAttribute, as demonstrated in the listing. Next, a shared instance of MyListener was added. Because the field, listener, is shared, we will only get (and we only need) one instance of the listener. The arbitrarily named Init and Deinit methods are run one time each, for each test. The SetUpAttribute indicates that Init should be run immediately before a test, and the TearDownAttribute indicates that the Deinit method should be run when a test has completed. These two methods add and remove the customer listener, respectively. To complete the class, I added some sample test methods, demonstrating some features of NUnit, such as the Test and Ignore attributes and NUnit's Assertion class. This pre-canned code will speed up writing test classes.
Make sure that you compile and test the prototype code. The results in NUnit should look something like those shown in Figure 1. (Keep in mind that the prototype intentionally fails or ignores, but green is the color we are shooting for in actual tests.)
Figure 1: Our prototype NUnit class works as coded.
Creating the Project Template Folder
Now that we have a prototype that we'd like to put into production, we need to start working on the assembly line. (Sorry, living in Michigan makes it hard not to use mass production metaphors.)
Several project and item templates already exist. The easiest way to create a new template is to start with an existing template. The closest relative to our template is the Visual Basic .NET Class Library template. To stub our project template, navigate to the ClassLibrary folder—which should be located approximately here: C:\Program Files\Microsoft Visual Studio .NET 2003\Vb7\VBWizards\ClassLibrary—and make a copy of this folder. Pasting the copy into the same parent folder—VBWizards—renames the copy TestClassLibrary.
Our new TestClassLibrary contains many of the elements we will need to create the new project item template.
Implementing the Template Class File
The ClassLibrary template contains a stubbed-out class module. To implement the NUnit class module, we need to replace the default class.vb module's code with templatized code from our class library prototype.
Note: Our new TestClassLibrary project folder contains two child folders. The first is scripts and the second is templates. Templates contains a sub-folder indicating the installed language version. English versions will be .\Templates\1033. It is the 1033 folder that contains the templatized class.vb module. The language version folder will vary depending on which language version of .NET you have installed.
To convert the prototype to a template, we can use the original class module as a guide. The vsWizard.dll uses substitution to replace specific tokens in the template file with values provided by VS.NET when we create a new project or add a new element. Listing 2 contains the original .\Templates\1033\class.vb module and Listing 3 contains the new code with replaceable template values for the elements replaced by VS.NET.
Listing 2: The ClassLibrary template class module.
Public Class [!output SAFE_ITEM_NAME] End Class
Listing 3: The TestClassLibrary template class module.
Imports NUnit.Framework Imports System.Diagnostics Friend Class MyListener Inherits TraceListener Public Overloads Overrides Sub Write(ByVal message As String) Console.Write(message) End Sub Public Overloads Overrides Sub WriteLine(ByVal message As String) Console.WriteLine(message) End Sub End Class <TestFixture()> _ Public Class [!output SAFE_ITEM_NAME] Private Shared listener As MyListener = New MyListener <SetUp()> _ Public Sub Init() If (Not Trace.Listeners.Contains(listener)) Then Trace.Listeners.Add(listener) End If End Sub <TearDown()> _ Public Sub Deinit() If (Trace.Listeners.Contains(listener)) Then Trace.Listeners.Remove(listener) End If End Sub <Test()> _ Public Sub Test1() Assertion.Assert("Add a boolean test here", False) End Sub <Test()> _ Public Sub Test2() Assertion.Fail("Intentionally fail if this code is run") End Sub <Ignore("Not ready yet!")> _ Public Sub Test3() ' Change the attribute from Ignore to Test ' when this test is finished End Sub End Class
Notice that the only part we needed to change was the name of the test class. Shown underlined, the class name needs to be a replaceable parameter because VS.NET uses a naming scheme classx.vb, where x is a number guaranteeing that the new file name is unique. Typically, the class module's name is class1.vb in a new project.
In Part 1 of this article, you learned how to combine NUnit and TraceListeners to create great test libraries. In addition, you learned a bit about how VS.NET creates new projects: This is accomplished with stubbed sample files, and you can use the same approach to create project or item templates.
The benefit of using this project template is that new developers can start creating NUnit tests right away and for every test we write we save 50 lines of code and the steps required to import the correct assemblies. It is the accumulative effect of short cuts like these that turn good programmers into hyper-productive programmers.
In Part 2 of this article, I will complete the project template by demonstrating how to modify the supporting script to import the NUnit namespaces, how to define the wizard launching file, add the VSDir file entries, and we will take our new test library, created by the template, for a spin around the block.
Paul Kimmel is the VB Today columnist for codeguru.com and developer.com and has written several books on object-oriented programming, including the recently released Visual Basic .NET Power Coding from Addison-Wesley and the upcoming Excel VBA 2003: Programmer's Reference from Wrox Press. He is the chief architect for Software Conceptions and is available to help design and build your next application.
The Lansing, Michigan area has started a new .NET Users Group. A well-run group, it offers great learning and networking opportunities and occasionally some free pizza and door prizes. Contact me at firstname.lastname@example.org if you live in mid-Michigan and are interested in participating.