Application Security Testing: An Integral Part of DevOps
Microsofts long-time practice of testing products in development internally, before they are finalized and released to market, was critical to building the companys next-generation development platform and tools. Both Microsoft Visual Studio 2010 and .NET Framework 4 were dogfooded extensively by the software giant at various development milestones throughout the product cycle, according to S. Somasegar, senior vice president, Developer Division. At this point in time, more than 10% of the Redmond companys employees are already running Microsoft Visual Studio 2010 into production environments.
Eating one's own dog food, also called dogfooding, is when a company uses the products that it makes. Dogfooding can be a way for a company to demonstrate confidence in its own products, and hence a kind of testimonial advertising. For example, Microsoft and Google emphasize the internal use of their own software products.
- The product's developers are familiar with using the products they develop.
- The company's members have direct knowledge and experience with its products.
- Users see that the company has confidence in its own products.
- Technically savvy users in the company are able to discover and report bugs in the products before they are released to the general public.
Microsoft teams have a long tradition of dogfooding their products long before releasing them to external customers for their own adoption and Developer Division is no exception to that. In fact, because we are in the same business as many of our customers (namely writing software products) I would say that dogfooding is an especially important practice for us to embrace. Im happy to tell you that with Microsoft Visual Studio 2010 we have redoubled our efforts in this area, Somasegar explained.
When building Microsoft Visual Studio 2010, Microsoft had all the developers leverage Team Foundation Server for tasks such as bug tracking and source control. Somasegar shared some numbers related to the feedback generated as the products were being dogfooded. It seems that toward the final part of the development process, feedback generated some 15.5 terabytes of data, no less than 896 builds were being compiled each month, with the team overseeing 828,978 work items, from bugs to change tasks.
The other big thrust of our dogfooding effort for the Microsoft Visual Studio 2010 and .NET Framework 4 project was to ensure that we had broad adoption of the latest builds of Microsoft Visual Studio itself. Unlike with TFS where the service is centrally managed, the tools developers and testers use on their desktops are not. As such, we built telemetry into dogfood builds of VS to help us understand how many folks were using the product daily and what builds were deployed throughout the team. As of today, we have over 12,000 people inside Microsoft running Microsoft Visual Studio 2010 for their day-to-day development. In addition, employees installed pre-release versions of .NET Framework 4 to their desktops more than 35,000 times, and Microsoft.com and MSDN deployed pre-release .NET Framework 4 on 112 servers, Somasegar added.
Both Microsoft Visual Studio 2010 and .NET Framework 4 were dogfooded extensively by the software giant at various development milestones throughout the product cycle.