MFC and .NET: Why ShellExecute Isn't Dependable and How to Work Around It

WEBINAR: On-demand webcast

How to Boost Database Development Productivity on Linux, Docker, and Kubernetes with Microsoft SQL Server 2017 REGISTER >

Welcome to this week's installment of .NET Tips & Techniques! Each week, award-winning Architect and Lead Programmer Tom Archer from the Archer Consulting Group demonstrates how to perform a practical .NET programming task using either C# or Managed C++ Extensions.

One of my primary goals in the .NET Tips & Techniques series is to provide practical advice that will help fellow coders save valuable time by offering my experiences as a programmer who's been using .NET since the very early betas. A prime example of such an experience was when I attempted to slowly migrate a few of my MFC applications to .NET by simply turning on the /CLR option and including the usage of a few choice .NET BCL (Base Class Library) classes. Much to my surprise (and chagrin), a very unlikely function call stopped working completely! That call was to the ShellExecute function. In many of my demo applications, I use ShellExecute from the About Box dialogs in order to allow the user to point his or her browser to the various support pages on my Web site.

The Problem

A bit of basic (process-of-elimination) debugging proved that ShellExecute failed only when the .NET methods were called. When those methods were removed, it worked. After a lot more debugging, I realized that the main issue had to do with the fact that the ShellExecute function needs to be called from an STA (Single-Threaded Apartment) thread when being passed a value that causes the browser to be executed. This is because if the ShellExecute function receives a value starting with "http://", it uses the Web Browser COM objects to launch the URL. (If you are launching an executable rather than a URL, then the ShellExecute function works correctly.)

This is not an issue with normal MFC applications because by default, the Visual Studio AppWizard generates MFC applications with a call to the AfxOleInit function from the application class's InitInstance member function. The AfxOleInit function then calls the COM function CoInitializeEx, specifying the STA theading model. However, mixing MFC and .NET brings the .NET CLR (Common Language Runtime) into the picture. When the CLR is invoked, it calls CoInitializeEx and sets the application's main thread to the MTA (Multi-Threaded Apartment) threading model, which is what ultimately causes the ShellExecute function to fail.

The Solution(s)

There are two solutions to this problem. The first involves performing the following steps in order to ensure that the application's main thread is an STA thread:

  1. Define a custom manage entry point for your application - /ENTRY:MyMain.
  2. Decorate the new entry point with the [System::STAThread] attribute.
  3. Call the WinMainCRTStartup function from the CRT entry point.

Here's what that would look like:

extern "C" void WinMainCRTStartup();

#using <System.dll>

void MyMain()

However, I personally found it much faster to simply call the System::Diagnostics::Process::Start method. After all, I do have access to the entire .NET BCL when I flip the CLR switch. This is by far the easiest way to solve this problem.

The Ins and Outs of a Workaround

This article was a long-winded way of saying "use the Process::Start method instead of ShellExecute when mixing MFC and .NET code." However, it's one thing to know what the workaround is and another to know why you have to perform a workaround in the first place.

So in keeping with the spirit of this series, hopefully the information you read here saves you the several hours I spent tracking all this down.

About the Author

The founder of the Archer Consulting Group (ACG), Tom Archer has been the project lead on three award-winning applications and is a best-selling author of 10 programming books as well as countless magazine and online articles.

About the Author

Tom Archer - MSFT

I am a Program Manager and Content Strategist for the Microsoft MSDN Online team managing the Windows Vista and Visual C++ developer centers. Before being employed at Microsoft, I was awarded MVP status for the Visual C++ product. A 20+ year veteran of programming with various languages - C++, C, Assembler, RPG III/400, PL/I, etc. - I've also written many technical books (Inside C#, Extending MFC Applications with the .NET Framework, Visual C++.NET Bible, etc.) and 100+ online articles.


  • problem with shellEXecute using Vista

    Posted by Manoj_Singh_JP007 on 10/19/2006 01:44am

    when i run PC Doctor 5 for windows in vista, it pops up a messagebox saying "A program needs ur permission to continue". When I try to run the same program from my application using shellExecute, it shows a messagebox saying "Administrative privileges required". My appliction is running with the following security peivileges... How do make the user see the messagebox saying "A program needs ur permission to continue" and let the user decide if he wants to continue running the application... in my case PC Doctor 5 for windows. Thanks

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Microsoft Azure® is a leading choice for businesses looking to take advantage of the cloud. Azure is particularly appealing to businesses that have already invested in Microsoft on-premises and are now considering running these applications and other workloads in the cloud. To understand how to make this move to Azure, many businesses are turning to managed service providers (MSPs) with specific Azure expertise. Read this white paper to learn the eight key areas to focus on when considering an MSP for an …

  • The software-defined data center (SDDC) and new trends in cloud and virtualization bring increased agility, automation, and intelligent services and management to all areas of the data center. Businesses can now more easily manage the entire lifecycle of their applications and services via the SDDC. This Aberdeen analyst report examines how a strong foundation in both the cloud and internal data centers is empowering organizations to fully leverage their IT infrastructure and is also preparing them to be able …

Most Popular Programming Stories

More for Developers

RSS Feeds

Thanks for your registration, follow us on our social networks to keep up-to-date