Extending Visual Studio 6.0 Debugger

Environment: VC6

Everybody has heard about the autoexp.dat file as a way to show custom data types in Visual Studio 6 debugger. It is very useful, but sometimes not satisfactory. For example, if during a debug session you want to see a variable value of some custom data type, e.g. FILETIME. FILETIME is 64-bit value which represents a number of 100 nano second units since January, 1st 1601. This value is divided into a struct containing two 32-bit values: Low and high 32 bits of that 64-bit value.

typedef struct _FILETIME {
  DWORD dwLowDateTime;
  DWORD dwHighDateTime;
} FILETIME, *PFILETIME, *LPFILETIME;

Using documented autoexp.dat capabilities is simply not possible to show date time (and many others custom data types) in a human readable form during a debug session. Until I discovered an undocumented autoexp.dat feature I had used a simple Visual Studio 6.0 add-in which was invoked from my add-in toolbar button. Of course, such solution did the job, but it is a little bit annoying to select a variable or set a mouse cursor to it in the source code window and then press the add-in toolbar button. I thought there is no way to show myself interpreted variable value in a debugger tooltip or watch window.

But, still, there is a way to do it. What you need to do is to write a WIN32 DLL which exports function with the following prototype:

HRESULT WINAPI AddIn_SystemTime( DWORD dwAddress,
                                 DEBUGHELPER *pHelper,
                                 int nBase,
                                 BOOL bUniStrings,
                                 char *pResult,
                                 size_t max,
                                 DWORD reserved )

I will explain the meaning of the parameters and data types soon. After you wrote that function which needs to evaluate your custom data type variable value, you just need to add the following line in your autoexp.dat (AutoExpand section) file:

MyDataType=$ADDIN(MyLibrary.dll,fCustomDataTypeViewer)

where MyDateType is a data type you are writting a custom viewer for, MyLibrary.dll is a dll from which you exported a custom viewer function and fCustomDataTypeViewer is the name of the exported function. It is best to put a full path to the DLL instead of DLL name only, because Visual Studio sometimes gets confused and this feature was not working properly when I put just a dll name without a path.

For FILETIME example it should look like this:

HRESULT WINAPI FILETIME_Viewer( DWORD dwAddress,
                                     DEBUGHELPER *pHelper,
                                     int nBase,
                                     BOOL bUniStrings,
                                     char *pResult,
                                     size_t max,
                                     DWORD reserved )
{
  FILETIME FileTime;
  SYSTEMTIME SysTime;
  DWORD nGot;
  if (pHelper->dwVersion<0x20000)
  {
     // This is the way to find out which debugger
     // version is running VC 6.0 version
     if (pHelper->ReadDebuggeeMemory(pHelper,
         dwAddress, sizeof(FileTime), &FileTime,
         &nGot)!=S_OK)
             return E_FAIL;

     if (nGot!=sizeof(FileTime))
             return E_FAIL;
  }
  else
  {
     if (pHelper->ReadDebuggeeMemoryEx(pHelper,
         pHelper->GetRealAddress(pHelper),
         sizeof(FileTime), &FileTime,&nGot)!=S_OK)
              return E_FAIL;
     if (nGot!=sizeof(FileTime))
              return E_FAIL;
  }
  // convert to SystemTime
  if (!FileTimeToSystemTime( &FileTime, &SysTime ))
     return E_FAIL;

  // and then just call the simple formatting function
  // which will fill the out buffer

  return FormatDateTime( &SysTime, pResult, max );
}

And now, here is the explanation of types and parameters used:

Each time when your mouse cursor is over some variable (or the variable is in a watch or a quick watch window) of FILETIME data type, the debugger will call your dll's exported function and if it returns S_OK, pResult contents will be shown in a debugger tooltip.

dwAddress is an address of your object in the debugger memory space. Of course, you can't use this address in your custom viewer DLL memory space. You need to read it using a callback function provided with the DEBUGHELPER structure which is declared on the following way:

typedef struct tagDEBUGHELPER
{
  DWORD dwVersion;
  BOOL (WINAPI *ReadDebuggeeMemory)(struct tagDEBUGHELPER *pThis,
                    DWORD dwAddr,
                    DWORD nWant, 
                    VOID* pWhere, 
                    DWORD *nGot );
  // from here only when dwVersion >= 0x20000
  DWORDLONG (WINAPI *GetRealAddress)( struct tagDEBUGHELPER *pThis );
  BOOL (WINAPI *ReadDebuggeeMemoryEx)( struct tagDEBUGHELPER *pThis,
                     DWORDLONG qwAddr,
                     DWORD nWant, 
                     VOID* pWhere, 
                     DWORD *nGot );
  int (WINAPI *GetProcessorType)( struct tagDEBUGHELPER *pThis );
} DEBUGHELPER;

typedef HRESULT (WINAPI *CUSTOMVIEWER)(DWORD dwAddress,
                                       DEBUGHELPER *pHelper,
                                       int nBase,
                                       BOOL bUniStrings,
                                       char *pResult,
                                       size_t max,
                                       DWORD reserved );

For Visual Studio 6.0 debugger version you are interested only in ReadDebuggeeMemory callback. This callback reads the object being debugged in memory into the memory structure provided with your custom viewer. It doesn't necessarily need to be the real object as is in your program. You just need to provide the matching memory layout for the object members that you are interested in.

The next parameter, DWORD nWant, represents a size in bytes you want to copy. VOID* pWhere is a memory where the object/part of the object being debugged will be copied. Finally, DWORD *nGot is an output value that shows how many bytes were copied successfully.

After you read the memory from the program debugged memory space, it is up to you to interpret it and return a result to the debugger. The resulting expression should be copied into char *pResult watching for the maximal buffer size which debugger has sent you through size_t max parameter. Also, there are additional parameters available:

int nBase which is hex or decimal system base of your current debugger setting and BOOL bUniStrings, which should be ignored.

You should be careful, because if your custom evaluator DLL crashes, Visual Studio session will also crash. Not only can one function can be exported from this DLL, but you can export as many functions as many different custom data type viewers you want to introduce.

As I know, this hidden feature is not documented in Visual Studio 6 companion documentation. It appears for Visual Studio .NET, but I tried it also in Visual Studio 6.0 and it works. You even don't need to have any part of Visual Studio .NET installed on your machine.

Of course, this feature is very suitable to evaluate and show some more compound data types, especially classes from your framework. I just gave a single example with a FILETIME object.

Downloads

Download demo project - 12 Kb
Download source - 3 Kb


Comments

  • Extending Visual Studio 6.0 Debugger

    Posted by Legacy on 01/23/2004 12:00am

    Originally posted by: gn chakrapani

    Hi All ,
    I used this in my code to read a string which can be either a wchar or char ,if the length of the string is more than 255 charectars the vc++ crashes and i see that it crashes even before my addin exported function is called

    have anybody faced the problem like this before if you have any solution for this give me


    Regards
    Pani

    • check max len

      Posted by vbbartlett on 03/22/2006 11:48am

      You are overwriting memory. the max var tells you the size of the string.

      Reply
    Reply
  • How do you deal with nested pointers and COM objects?

    Posted by Legacy on 09/12/2002 12:00am

    Originally posted by: Chango

    The $ADDID auto-expansion mechanism seems to work only for primitive, non-pointer datatypes and simple structures. If you make an entry for an object that contains pointers to something else, these pointers appear to be invalid in the address space of the DLL. I guess calling ReadDebuggeeMemory() on these pointers could fetch the right data from the debugged process, but that would be difficult to do and not even always possible. And when it comes to COM object pointers, I have no clue how to "marshal" them. That's exactly what I'd like to be able to do. (It's so easy to look at all the properties of an object in VB and so difficult in VC++...) Anyone have an idea??

    // Chango V.

    Reply
  • DATE display

    Posted by Legacy on 07/30/2002 12:00am

    Originally posted by: Stan

    It is very interesting information. Now I want to display in user readable format for either DATE structure or COleDateTime class. I wrote function for DATE and added entry in AUTOEXP.DAT:

    DATE=$ADDIN(C:\AddIn\Release\CustomEvaluator.dll,DATE_Evaluate)
    (I tried _DATE with the same result)

    However it doesn't seem to work. THe function is properly implemented and exported. I tried using SYSTEMTIME as in the sample and it works. The difference I see is SYSTEMTIME is a structure while DATE is a typedef for double.
    I suspect problem with definition in AUTOEXP.DAT
    Anybody has a clue ?

    Cheers,
    Stan

    Reply
  • Great, but not for all types

    Posted by Legacy on 04/18/2002 12:00am

    Originally posted by: Stephan Pilz

    Very good.

    I tried to see my timt_t values. But it doesn't work. The Problem is in autoexp.dat. If I write the type timt_t (or _time_t) my Evaluatefunc is'nt called. If I link my Function to the Type SYSTEMTIME, its called.

    What's the problem?

    Reply
  • Nice article

    Posted by Legacy on 03/25/2002 12:00am

    Originally posted by: Jack Hendriks

    Hello,

    I found your article very good and explaining matters very well.

    But when I started with your example and test project I could not get it to work
    Finally I discovered that in the "autoexp.dat" file the type had to be "_FILETIME" instead of "FILETIME".

    With this minor error corrected all worked well.

    Thanks and perhaps until a next time,

    Jack Hendriks

    Reply
  • Very cool!

    Posted by Legacy on 02/28/2002 12:00am

    Originally posted by: Thomas S�ldenwagner


    Thanks alot for your posting!

    It bored me for years! There is the solution!

    THX a lot!

    Reply
  • Very Good

    Posted by Legacy on 02/26/2002 12:00am

    Originally posted by: Sein

    Thanks

    Reply
  • Excellent.

    Posted by Legacy on 02/25/2002 12:00am

    Originally posted by: Jignesh

    Great work! :)

    Reply
Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Live Event Date: November 20, 2014 @ 2:00 p.m. ET / 11:00 a.m. PT Are you wanting to target two or more platforms such as iOS, Android, and/or Windows? You are not alone. 90% of enterprises today are targeting two or more platforms. Attend this eSeminar to discover how mobile app developers can rely on one IDE to create applications across platforms and approaches (web, native, and/or hybrid), saving time, money, and effort and introducing apps to market faster. You'll learn the trade-offs for gaining long …

  • Live Event Date: October 29, 2014 @ 11:00 a.m. ET / 8:00 a.m. PT Are you interested in building a cognitive application using the power of IBM Watson? Need a platform that provides speed and ease for rapidly deploying this application? Join Chris Madison, Watson Solution Architect, as he walks through the process of building a Watson powered application on IBM Bluemix. Chris will talk about the new Watson Services just released on IBM bluemix, but more importantly he will do a step by step cognitive …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds