Hooking a DirectX/COM Interface

After all the helpful articles I read here, I am glad that I can contribute to a subject that has not yet been covered.

This article features a description on how to hook a DirectX/COM Interface. I used the DirectInput interface as an example of how to hook an interface function. For the basic Windows hook, I referred to an article by Wade Brainerd that describes the API hooking process.

Task

Intercepting a method of a COM Interface requires an extended approach compared to hooking an API call. If the desired DLL is examined, only the create Interface function is actually exported by the DLL. So, how can you hook your desired function?

A COM interface is basically a list of virtual function pointers that are linked together. You merely have to follow the links and modify every node until you finally reach the pointer of the function that you want to replace.

Step 1

As you can see, only the create interface COM functions are visible, so you have to start your hooking chain at the DirectInputCreate function, which returns a COM interface. Here, you have to inject your DLL into the import address table (IAT) of the calling program.

Step 2

If the calling program invokes a DirectInputCreate, your function is called. Also, you receive a pointer to a pointer of a pointer to a virtual function table that is the interface of the direct input.

DECLARE_INTERFACE_(IDirectInputW, IUnknown)
{
   /*** IUnknown methods ***/
   STDMETHOD(QueryInterface)(THIS_ REFIID riid,
                             LPVOID * ppvObj) PURE;
   STDMETHOD_(ULONG,AddRef)(THIS) PURE;
   STDMETHOD_(ULONG,Release)(THIS) PURE;

   /*** IDirectInputW methods ***/
   STDMETHOD(CreateDevice)(THIS_ REFGUID,LPDIRECTINPUTDEVICEW *,
                           LPUNKNOWN) PURE;
   STDMETHOD(EnumDevices)(THIS_ DWORD,LPDIENUMDEVICESCALLBACKW,
                          LPVOID,DWORD) PURE;
   STDMETHOD(GetDeviceStatus)(THIS_ REFGUID) PURE;
   STDMETHOD(RunControlPanel)(THIS_ HWND,DWORD) PURE;
   STDMETHOD(Initialize)(THIS_ HINSTANCE,DWORD) PURE;
};

Step 3

Now, you can create your device with CreateDevice. You again will receive an address to a different virtual function pointer table; it represents the Device.

Pick the method you want to replace and change the virtual function pointer table in the appropriate place to inject your function.

Step 4

Do the actual data manipulation.

Implementation

Step 1

To hook yourself into an API function, you simply can use the Windows API call SetWindowsHookEx. Here, you create a system hook to monitor the starting processes and match them to your desired program. After you have identified your program, you have to compare the import module names with the DLL you want to replace. Because this hook is written for direct input, the entry you want is:

DINPUT8.DLL

To find this entry, you have to loop through the descriptors until you find your DLL.

// Iterate through each import descriptor, and redirect if
// appropriate
   while ( pImportDesc->FirstThunk )
   {
      PSTR pszImportModuleName = MakePtr( PSTR, hModEXE,
                                          pImportDesc->Name);

      if ( lstrcmpi( pszImportModuleName, Hook->Name ) == 0 )
      {
         sprintf(dbBuffer,"Dll Found in module  %s replace it\n",
                 Hook->Name );
         WriteToLog(dbBuffer);
         RedirectIAT( Hook, pImportDesc, (PVOID)hModEXE );
      }

      pImportDesc++;    // Advance to next import descriptor
   }

After you find your entry, you have to remove the write protection from the IAT with

VirtualQuery( pIAT, &mbi, sizeof(mbi) );

to be able to write into the memory. After the memory is open, you have to find your entry by iterating through the IAT.

while ( pIteratingIAT->u1.Function )
{
   void* HookFn = 0;    // Set to either the SFunctionHook or pStubs.

   // import by name
   if ( !IMAGE_SNAP_BY_ORDINAL( pINT->u1.Ordinal ) )
   {
      PIMAGE_IMPORT_BY_NAME pImportName =
         MakePtr( PIMAGE_IMPORT_BY_NAME, pBaseLoadAddr,
                  pINT->u1.AddressOfData );

      // Iterate through the hook functions, searching for this import.
      SFunctionHook* FHook = DLLHook->Functions;
      while ( FHook->Name )
      {
         if ( lstrcmpi( FHook->Name, (char*)pImportName->Name ) == 0 )
         {
            sprintf(dbBuffer,"Hooked function: %s\n",
                    (char*)pImportName->Name );
            WriteToLog(dbBuffer);
            // Save the old function in the SFunctionHook structure
            // and get the new one.
            FHook->OrigFn = (unsigned long*)pIteratingIAT->u1.Function;
            HookFn = FHook->HookFn;
            break;
         }

         FHook++;
      }

   }
}

Now, you can now replace it with your own.

// Replace the IAT function pointer if we have a hook.
if ( HookFn )
{
   // Cheez-o hack to see if what we're importing is code or data.
   // If it's code, we shouldn't be able to write to it
   if ( IsBadWritePtr( (PVOID)pIteratingIAT->u1.Function, 1 ) )
   {
      pIteratingIAT->u1.Function = (DWORD)HookFn;
   }
   else if ( osvi.dwPlatformId == VER_PLATFORM_WIN32_WINDOWS )
   {
      // Special hack for Win9X, which builds stubs for imported
      // functions in system DLLs (Loaded above 2GB). These stubs are
      // writeable, so we have to explicitly check for this case
      if ( pIteratingIAT->u1.Function > (DWORD)0x80000000 )
         pIteratingIAT->u1.Function = (DWORD)HookFn;
   }
}

The only thing remaining is to restore the memory attributes, as though nothing ever happened.

VirtualProtect( pIAT, sizeof(PVOID) * cFuncs, flOldProtect, &flDontCare);

Hooking a DirectX/COM Interface

Step 2

Inside of the CreateInterface method, you start hooking into the COM interface by injecting your own CreateDevice function pointer into the Virtual function table (Vtbl), which is returned in the ppvOut pointer of the original call.

DirectInput8Create_Type OldFn =
   (DirectInput8Create_Type)D3DHook.Functions
      [D3DFN_DirectInput8Create].OrigFn;
      HRESULT hr = OldFn( hinst, dwVersion, riidltf, ppvOut,
                          punkOuter );

Resolve the pointer until you get the pointer to the Vtbl of the interface.

With this address, you again have to remove the memory protection before you can inject your function into the table and save the old function pointer for later use. Inject your function pointer into the offset of the CreateDevice function pointer inside of the interface Vtbl and restore the memory protection.

As you can see, the CreateDevice is the fourth method of the DirectInput interface; this means the offset inside of the Vtbl is 0x0C (pointer(DWORD)*index 3).

typedef struct IDirectInput *LPDIRECTINPUT;

#if !defined(__cplusplus) || defined(CINTERFACE)
#define IDirectInput_QueryInterface(p,a,b)
        (p)->lpVtbl->QueryInterface(p,a,b)
#define IDirectInput_AddRef(p) (p)->lpVtbl->AddRef(p)
#define IDirectInput_Release(p) (p)->lpVtbl->Release(p)
#define IDirectInput_CreateDevice(p,a,b,c)
        (p)->lpVtbl->CreateDevice(p,a,b,c)
#define IDirectInput_EnumDevices(p,a,b,c,d)
        (p)->lpVtbl->EnumDevices(p,a,b,c,d)
#define IDirectInput_GetDeviceStatus(p,a)
        (p)->lpVtbl->GetDeviceStatus(p,a)
#define IDirectInput_RunControlPanel(p,a,b)
        (p)->lpVtbl->RunControlPanel(p,a,b)
#define IDirectInput_Initialize(p,a,b)
        (p)->lpVtbl->Initialize(p,a,b)
#else
#define IDirectInput_QueryInterface(p,a,b)
        (p)->QueryInterface(a,b)
#define IDirectInput_AddRef(p) (p)->AddRef()
#define IDirectInput_Release(p) (p)->Release()
#define IDirectInput_CreateDevice(p,a,b,c) (p)->CreateDevice(a,b,c)
#define IDirectInput_EnumDevices(p,a,b,c,d) (p)->EnumDevices(a,b,c,d)
#define IDirectInput_GetDeviceStatus(p,a) (p)->GetDeviceStatus(a)
#define IDirectInput_RunControlPanel(p,a,b) (p)->RunControlPanel(a,b)
#define IDirectInput_Initialize(p,a,b) (p)->Initialize(a,b)
#endif

After you know where to inject it, you can start thinking about the implementation. When you look at the declaration of CreateDevice in dinput.h, it does not match up with the declaration you see in the DirectX Help.

HRESULT CreateDevice(

   REFGUID rguid,
   LPDIRECTINPUTDEVICE *lplpDirectInputDevice,
   LPUNKNOWN pUnkOuter
);

As you can see in the definition inside the dinput.h, you have to add a fourth parameter, the interface pointer. This ends up in the following function declaration.

HRESULT __stdcall   PASCAL MyCreateDevice(LPVOID *ppvOut,
                                          REFGUID rguid,
                                          LPDIRECTINPUTDEVICE
                                          *lplpDirectInputDevice,
                                          LPUNKNOWN pUnkOuter)

This is important. You have to make sure to use the __stdcall calling convention in your declaration; refer to the MSDN.

The __stdcall calling convention is used to call Win32 API functions. The callee cleans the stack. __cdecl is the default calling convention for C and C++ programs. The stack has to be cleaned up by the caller, which is not the case with your function.

When you look at the disassembly of this call, you can see the stack pointer verification function _RTC_CheckEsp is called after the call to the interface function.

if (lpdi->CreateDevice(GUID_SysKeyboard, &lpdikey, NULL)!=DI_OK)
00401365  mov         esi,esp
00401367  push        0
00401369  push        offset lpdikey (4552C8h)
0040136E  push        offset _GUID_SysKeyboard (44643Ch)
00401373  mov         eax,dword ptr [lpdi (4552C4h)]
00401378  mov         ecx,dword ptr [eax]
0040137A  mov         edx,dword ptr [lpdi (4552C4h)]
00401380  push        edx
00401381  mov         eax,dword ptr [ecx+0Ch]
00401384  call        eax
00401386  cmp         esi,esp
00401388  call        _RTC_CheckEsp (4026A0h)
0040138D  test        eax,eax
0040138F  je          Game_Init+78h (401398h)
   return(0);
00401391  xor         eax,eax
00401393  jmp         Game_Init+107h (401427h)

// set cooperation level
if (lpdikey->SetCooperativeLevel(main_window_handle,

If you forget to declare your function with __stdcall, your function will process fine, but you will fail the esp pointer test of this function; it sets the eax and tests it after the function call.

Step 3

When you create the device now, the call is re-routed to your CreateDevice function. After you call the original function, you receive a new pointer in lplpDirectInputDevice, which will direct you to the Vtbl of the device.

HRESULT hr = OldCreateDev(ppvOut,rguid,lplpDirectInputDevice,pUnkOuter);

In my sample, I replaced the GetDeviceState because I can add additional input in the return data of the calling function. To get to the offset, you have to look at the definition inside the DInput.dll. You see that the GetDeviceState function is the tenth method inside of the Device, which leads to an offset of 0x24.

After you know the offset, you can proceed with the instructions in Step 2 to remove the protection, store the old pointer, inject your own function, and restore the memory protection.

Step 4

When the GetDeviceState function is called by the target program, the injected function is called and you can manipulate the data as you wish.



About the Author

Martin Mueller

My first PC program was written on MS DOS in Assembler, after I got Peter Norton's Assembler book where the Edlin and DOS Debugger supported me in exploring my and code of others. I went from there over IDE/Debugger like Borland Turbo Debugger over several microcontroller H8, ST16, 8051, SLE66, SLE88 to my favorite MS Visual Studio 6. Now I mainly program in C++ and Java, where I prefer C++ since your abilities to interfere with the system are an easier task, which is still my favorite, than implementing some natives in Java to enhance the functionality. I am working since 18 years as a programmer and still love every day of it. My experience include ever level of programming from Hardware programming, where you have to take care of nano seconds in your assembler program over real time operating system programming, to VM programming, system services, applications, game programming, web server to high level web script programming.

Downloads

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

  • Wednesday, September 24, 2014 8:00 AM - 9:00 AM PDT According to a recent Forrester Research report, many companies are choosing low-code platforms over traditional programming platforms, due to the speed with which low-code apps can be assembled and tested. With customer-facing applications on the rise, traditional programming platforms simply can't keep up with the "short schedules and rapid change cycles" required to develop these applications. Check out this upcoming webinar and join Clay Richardson from …

  • Live Event Date: September 16, 2014 @ 11:00 a.m. ET / 8:00 a.m. PT Are you starting an on-premise-to-cloud data migration project? Have you thought about how much space you might need for your online platform or how to handle data that might be related to users who no longer exist? If these questions or any other concerns have been plaguing you about your migration project, check out this eSeminar. Join our speakers Betsy Bilhorn, VP, Product Management at Scribe, Mike Virnig, PowerSucess Manager and Michele …

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds