SetWindowsHookEx Questions

[avejidah]

Hi, thanks for taking the time to look at my questions. I am trying to use SetWindowsHookEx to filter window creation. My hook gets installed correctly but doesn't work quite as planned.
This program sets my hook:

#define _CRT_SECURE_NO_WARNINGS 1
#include <iostream>
#include <Windows.h>
#include <commctrl.h>
using namespace std;

int main()
{
HMODULE hinstDLL = LoadLibrary("testdll3.dll");
cout << "hinstDLL: " << hinstDLL << endl;
HOOKPROC CBTProc = (HOOKPROC)GetProcAddress(hinstDLL, "MyCBTProc");
cout << "CBTProc: " << CBTProc << endl;
HHOOK CBTHook = SetWindowsHookEx(WH_CBT, CBTProc, hinstDLL, 0);
system("pause");
//UnhookWindowsHookEx(CBTHook);
return 0;
}


Here's the dll:

#include <iostream>
#include <Windows.h>
#include <commctrl.h>
#include <Psapi.h>
using namespace std;


#ifdef _MANAGED
#pragma managed(push, off)
#endif

#define PLUGIN_EXPORT extern "C" __declspec(dllexport)

PLUGIN_EXPORT LRESULT MyCBTProc(int code, WPARAM wParam, LPARAM lParam)
{
if (code < 0)
return CallNextHookEx(0, code, wParam, lParam);

if (code == HCBT_CREATEWND)
{
char fileName[100];
HANDLE hndProc = GetCurrentProcess();
GetModuleFileNameEx(hndProc, NULL, fileName, 100);
if (strstr(fileName, "taskmgr.exe"))
Beep(750, 50);
return 0;
}
//CallNextHookEx(0, code, wParam, lParam);
return 0;
}

BOOL APIENTRY DllMain(HMODULE hModule,
DWORD ul_reason_for_call, LPVOID lpReserved)
{
return true;
}

#ifdef _MANAGED
#pragma managed(pop)
#endif


I used task manager to test the hook. When task manager opens I hear a beep for each window created, which is what I would expect.

Questions:
1) Since the CBTProc function resides in a dll I figured that the function would stick around after my program finishes executing (I have UnhookWindowsHookEx commented out), however as soon as my program is finished the hook no longer works. I attached my debugger to taskmanager, and I see that the function is cleared from memory when my program exits. I also found with my debugger that my program calls UnhookWindowsHookEx automatically when returns control to the os. I don't understand why the dll doesn't stay injected after my program exits?

2) Even if I call UnhookWindowsHookEx manually the hook chain still seems messed up. If I run my program it works the first time - I hear the beeps. However, if I run it a second time every process in windows crashes and I have to do a full restart on the system to get the program to work again. Why is this happening? Am I missing something obvious?

Thanks.

[Igor Vartanov]

From MSDN:
Syntax

LRESULT CALLBACK CBTProc( int nCode,
WPARAM wParam,
LPARAM lParam
);

PLUGIN_EXPORT LRESULT MyCBTProc(int code, WPARAM wParam, LPARAM lParam)
Any difference?

Since the CBTProc function resides in a dll I figured that the function would stick around after my program finishes executingYou hoped in vain.

From MSDN:
Before terminating, an application must call the UnhookWindowsHookEx function to free system resources associated with the hook.Otherwise Windows takes care itself about freeing hooks nobody needs anymore.

[avejidah]

Thanks for the help, Igor. I didn't use CALLBACK because it makes my function a __stdcall, and then the function name seems to get mangled. I looked up __declspce(dllexport) on msdn and it says to use a __cdecl. If I use CALLBACK then my GetProcAddress(...) call fails. With the __cdecl the address is found and the function gets called.

Thanks for clearing up the Unhook. Makes sense.

Any idea why the hook chain seems to be messed up after the Unhook? If I run the program a second time every process crashes and I have to restart the pc, which leads me to believe that the hook chain is messed up.

Thanks again!

[Igor Vartanov]

Thanks for the help, Igor. I didn't use CALLBACK because it makes my function a __stdcall, and then the function name seems to get mangled. I looked up __declspce(dllexport) on msdn and it says to use a __cdecl. If I use CALLBACK then my GetProcAddress(...) call fails. With the __cdecl the address is found and the function gets called.First of all, you never need to implement it this way, I mean, export hook proc. Just place your hook/unhook code into the dll, export them the way you like and link to the dll with import library. The main point of exporting is an intention to call the function. You see? You never call the hook procedure, it's Windows who does, and Windows never requires it be exported.

Second, in case you insist on exporting, unmangle the name by means of .def file or look for mangled name while getting proc address... Anyhow, your hook procedure must be __stdcall, don't you see that from MSDN???

Thanks for clearing up the Unhook. Makes sense.You welcome.

Any idea why the hook chain seems to be messed up after the Unhook? If I run the program a second time every process crashes and I have to restart the pc, which leads me to believe that the hook chain is messed up.Your stack is corrupted because of illegal calling convention, so what good comes from this? Just make the things right from the very beginning.

[avejidah]

Awesome, thanks for clearing that up for me.