Working Around a VS .NET 2003 DDE Bug

To create applications with DDE support, you have to enter a file extension that will be associated with the document type by calling the CWinApp member RegisterShellFileTypes.

This allows the application to register a document type and extension in the Registry database, allowing you to open the document by double-clicking the filename with a registered extension in Windows Explorer. it also allows for other shell commands, such as printing the document from Explorer.

Unfortunately, a code modification between versions 7.0 and 7.1 of Visual Studio introduced a bug that makes this feature inoperable. If you compile a program using VS 7.1 (.NET 2003) and try opening the file using a DDE mechanism, you will encounter no error message at all or the message:

Error: failed to execute DDE command” with no further explanation.

Sometimes, you can see that the process is not terminated and still running after that error and Task menager has to be used to terminate the application.

The code change was supposed to prevent a buffer overrun. See code below.

Microsoft coders somehow forgot abut one line of code that would copy a DDE command from a LPCTSTR string retrieved by UnpackDDElParam, to a TCHAR string used as a parameter in a call to a CWinApp virtual member, OnDDECommand. Therefore, the DDE command is always empty and the DDE fails to open or print the file.

We have two possibilities: Override OnDDECommand or handle message WM_DDE_EXECUTE. I have chosen the second one so the existing code will not be executed. Because the string is passed to the WM_DDE_EXECUTE handler and should be passed to OnDDECommand, overriding the WM_DDE_EXECUTE handler seems to be more appropriate.

To do so:

Add #include <dde.h> to your StdAfx.h file for a WM_DDE_EXECUTE definition.

In the CMainFrame class, insert a mapping macro:

BEGIN_MESSAGE_MAP(CMainFrame, CMDIFrameWnd)
   ...
   ON_MESSAGE(WM_DDE_EXECUTE, OnDDEExecute)
END_MESSAGE_MAP()

And the definition of OnDDEExecute:

LRESULT CMainFrame::OnDDEExecute(WPARAM wParam, LPARAM lParam)
{
   // unpack the DDE message
   UINT_PTR unused;
   HGLOBAL hData;
   //IA64: Assume DDE LPARAMs are still 32-bit
   VERIFY(UnpackDDElParam(WM_DDE_EXECUTE, lParam, &unused,
          (UINT_PTR*)&hData));

   // get the command string
   TCHAR szCommand[_MAX_PATH * 2] = {0};
   LPCTSTR lpsz = (LPCTSTR)GlobalLock(hData);
   int commandLength = lstrlen(lpsz);

   // This line is added to original MS code.
   int arrayLen = sizeof(szCommand)/sizeof(TCHAR);

   // This line is changed to avoid _countof (another include file).
   if (commandLength >= arrayLen)
   {
      // The command would be truncated. This could be a security
      // problem.
      TRACE0("Warning: Command was ignored because it was too
              long.n");
      return 0;
   }

   // This line is needed to rectify a problem.
   lstrcpyn(szCommand, lpsz, arrayLen);
   GlobalUnlock(hData);

   // acknowledge now - before attempting to execute
   ::PostMessage((HWND)wParam, WM_DDE_ACK, (WPARAM)m_hWnd,
   //IA64: Assume DDE LPARAMs are still 32-bit
   ReuseDDElParam(lParam, WM_DDE_EXECUTE, WM_DDE_ACK,
                  (UINT)0x8000, (UINT_PTR)hData));

   // don't execute the command when the window is disabled
   if (!IsWindowEnabled())
   {
      TRACE(traceAppMsg, 0, _T("Warning: DDE command '%s' ignored
                                because window is disabled.n"),
            szCommand);
      return 0;
   }

   // execute the command
   if (!AfxGetApp()->OnDDECommand(szCommand))
      TRACE(traceAppMsg, 0, _T("Error: failed to execute DDE
                                command '%s'.n"), szCommand);

   return 0L;
}

In the header file:

afx_msg LRESULT OnDDEExecute(WPARAM wParam, LPARAM lParam);

The preceding change properly copies the string that is passed to OnDDECommand; now, DDE works like a charm.

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read