[RESOLVED] Hook Not Working

[Chazwazza]

Hello, I am trying to get the following code to work, but the HookProc is never called. Can anyone please help me, I can provide the project files or the rest of the project which I am using to call the code if absolutely necessary.

#include <windows.h>
#include <cstdio>
#include "main.h"

static HHOOK KeyHook = NULL;
static HWND MyHwnd = NULL;
static HINSTANCE g_hInstance = NULL;

BOOL WINAPI DllMain(HINSTANCE hInstance, DWORD fdwReason, LPVOID lpvReserved)
{
switch (fdwReason)
{
case DLL_PROCESS_ATTACH:
case DLL_PROCESS_DETACH:
UnhookWindowsHookEx(KeyHook);
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
UnhookWindowsHookEx(KeyHook);
break;
}

g_hInstance = hInstance;

return TRUE;
}


BIGBROTHER_API void InstallHook(HWND h)
{
KeyHook = NULL;
MyHwnd = h;
KeyHook = SetWindowsHookEx(WH_KEYBOARD,HookProc,g_hInstance,0);
if(KeyHook == NULL)
{
MessageBox(NULL,"Unable to install hook","Error!",MB_OK);
}
}

BIGBROTHER_API void RemoveHook()
{
UnhookWindowsHookEx(KeyHook);
}

BIGBROTHER_API LRESULT CALLBACK HookProc(int ncode,WPARAM wparam,LPARAM lparam)
{
if(ncode>=0)
{
if(lparam & (1 << 31))
{
SendMessage(MyHwnd,WM_USER+755,wparam,lparam);
}
}
return (CallNextHookEx(KeyHook,ncode,wparam,lparam) );
}

[UnfitElf]

I havent done much work (infact you could say none) with .dll files however,

case DLL_PROCESS_ATTACH:
case DLL_PROCESS_DETACH:
UnhookWindowsHookEx(KeyHook);
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
UnhookWindowsHookEx(KeyHook);
break;
doesnt look right as far as i (dont) know :P
When a process or thread attaches its going to call UnhookWindowsHookEx() this doesnt seem right...

Also if the code above is right then why not go

case DLL_PROCESS_ATTACH:
case DLL_PROCESS_DETACH:
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
UnhookWindowsHookEx(KeyHook);
break;
??

I would assume you would be wanting somthing more like

case DLL_PROCESS_ATTACH:
// call hook window here, or do nothing
break; // <-------------!!!
case DLL_PROCESS_DETACH:
UnhookWindowsHookEx(KeyHook);
break; // <-------------!!!
case DLL_THREAD_ATTACH:
// call hook window here, or do nothing
break; // <-------------!!!
case DLL_THREAD_DETACH:
UnhookWindowsHookEx(KeyHook);
break;


Hope this helps

[Chazwazza]

Sorry, I've tried your suggestions but the HookProc is still never being called. There must be something wrong, I just can't see it. Hopefully someone can find my mistake.

Thanks.

[UnfitElf]

Could you also post the code that shows where you are calling the InstallHook function.

Thanks

[Chazwazza]

This is the related parts (it's a large project), basically I create a window, then send the Hwnd of the window to the DLL when I install the hook, so it can use it to send messages back to the original window for processing (But the HookProc never gets called so this point).

HINSTANCE hInst;
HWND HwndMe;
TCHAR *szTitle = "MyProject";
TCHAR *szWindowClass = "MyProject";
BOOL InitInstance(HINSTANCE hInstance, int nCmdShow);
LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam);

int APIENTRY WinMain(HINSTANCE hInstance,HINSTANCE hPrevInstance,LPSTR lpCmdLine,int nCmdShow)
{
InitInstance(hInstance, nCmdShow);

InstallHook(HwndMe)

MSG msg;
while (GetMessage(&msg, NULL, 0, 0) > 0) {
TranslateMessage(&msg);
DispatchMessage(&msg);
}
return msg.wParam;

}

BOOL InitInstance(HINSTANCE hInstance, int nCmdShow)
{
/* Declare the necessary variabes */
HWND hWnd;
WNDCLASS wc;
hInst = hInstance;

/* Create a window class */
wc.style = 0;
wc.lpfnWndProc = WndProc;
wc.cbClsExtra = 0;
wc.cbWndExtra = 0;
wc.hInstance = hInstance;
wc.hIcon = NULL;
wc.hCursor = NULL;
wc.hbrBackground = (HBRUSH)(COLOR_WINDOW+1);
wc.lpszMenuName = NULL;
wc.lpszClassName = szWindowClass;

// Register the window class
RegisterClass(&wc);

// Create a message only window
hWnd = CreateWindow(szWindowClass, szTitle, WS_OVERLAPPEDWINDOW,
CW_USEDEFAULT, 0, CW_USEDEFAULT, 0, NULL, NULL, hInstance, NULL);

/* If creating the handle doesn't work, return with a failure message */
if (!hWnd)
{
return FALSE;
}

// Show the created window
ShowWindow(hWnd,SW_HIDE);

// Set the window's handle to our global variable
HwndMe = hWnd;

// Return with a success message
return TRUE;
}

/* Catch the messages sent to the window */
LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
switch(msg)
{
// If a keystroke message is recieved
case WM_KEYSTROKE:
// Call the function to process the keystrokes
ProcessKeys(wParam, lParam);
return TRUE;
break;
// If the program is being closed, then destroy the program's window
case WM_CLOSE:
DestroyWindow(hwnd);
break;
// When the window is destroyed, post a quit message to close the program
case WM_DESTROY:
PostQuitMessage(0);
break;
default:
// Calls the default window procedure to provide default processing for any window messages that an application does not process
return DefWindowProc(hwnd, msg, wParam, lParam);
}
return 0;
}

[UnfitElf]

ok because ive never actually done a Dll myself before, ive only seen about them, i thought it was time to give it a simple try.

Ive got it calling 2 functions, InstallHook and RemoveHook. Note that both these function only display MessageBoxes, they do NOT actuaully attempt to install a hook. Ill leave that part to you.

Here is the sorce:
main.cpp (the window)
//---------------------------------------------------------------------------------

// DLL example. i.e. calling a function from a .dll

//---------------------------------------------------------------------------------

// main.cpp

//---------------------------------------------------------------------------------

#include <windows.h>

//---------------------------------------------------------------------------------

#define DLLIMPORT __declspec(dllimport)

DLLIMPORT void InstallHook(HWND hOwner);
DLLIMPORT void RemoveHook();

//---------------------------------------------------------------
// Globals

HINSTANCE hInst;
HWND HwndMe;
TCHAR *szTitle = "MyProject";
TCHAR *szWindowClass = "MyProject";

//---------------------------------------------------------------
// Windows setup

LRESULT CALLBACK WindowProcedure (HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam);

int WINAPI WinMain (HINSTANCE hThisInstance,
HINSTANCE hPrevInstance,
LPSTR lpszArgument,
int nFunsterStil)

{
//HWND hwnd;
MSG messages; // Here messages to the application are saved
WNDCLASSEX wincl; // Data structure for the windowclass

// The Window structure
wincl.hInstance = hThisInstance;
wincl.lpszClassName = szWindowClass;
wincl.lpfnWndProc = WindowProcedure; // This function is called by windows
wincl.style = CS_DBLCLKS; // Catch double-clicks
wincl.cbSize = sizeof (WNDCLASSEX);

// Use default icon and mouse-pointer
wincl.hIcon = LoadIcon (NULL, IDI_APPLICATION);
wincl.hIconSm = LoadIcon (NULL, IDI_APPLICATION);
wincl.hCursor = LoadCursor (NULL, IDC_ARROW);
wincl.lpszMenuName = NULL; // No menu
wincl.cbClsExtra = 0; // No extra bytes after the window class
wincl.cbWndExtra = 0; // structure or the window instance
// Use Windows's default color as the background of the window
wincl.hbrBackground = (HBRUSH) COLOR_BACKGROUND;

// Register the window class, and if it fails quit the program
if (!RegisterClassEx (&wincl))
return 0;

// The class is registered, let's create the program
HwndMe = CreateWindowEx (
0, // Extended possibilites for variation
szWindowClass, // Classname
szTitle, // Title Text
WS_OVERLAPPEDWINDOW, // default window
CW_USEDEFAULT, // Windows decides the position
CW_USEDEFAULT, // where the window ends up on the screen
300, // The programs width
300, // and height in pixels
HWND_DESKTOP, // The window is a child-window to desktop
NULL, // No menu
hThisInstance, // Program Instance handler
NULL // No Window Creation data
);

hInst = hThisInstance;

// Make the window visible on the screen
ShowWindow (HwndMe, nFunsterStil);

InstallHook(HwndMe);

// Run the message loop. It will run until GetMessage() returns 0
while (GetMessage (&messages, NULL, 0, 0))
{
// Translate virtual-key messages into character messages
TranslateMessage(&messages);
// Send message to WindowProcedure
DispatchMessage(&messages);
}
// The program return-value is 0 - The value that PostQuitMessage() gave
return (int)messages.wParam;
}

//------------------------------------------------------------------------------
// Windows proc

LRESULT CALLBACK WindowProcedure (HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
{
switch (message)
{
case WM_CREATE:
break;
case WM_CLOSE:
case WM_ENDSESSION:
RemoveHook();
DestroyWindow(hwnd);
break;
case WM_DESTROY:
PostQuitMessage(0);
break;
default:
return DefWindowProc (hwnd, message, wParam, lParam);
}
return 0;
}

//------------------------------------------------------------------------------

And here is the .dll file sorce
//------------------------------------------------

#include <windows.h>

//------------------------------------------------

#define DLLEXPORT __declspec(dllexport)

//------------------------------------------------

static HWND MyHwnd = NULL;
static HHOOK KeyHook = NULL;
static HINSTANCE g_hInstance = NULL;

//------------------------------------------------

DLLEXPORT void InstallHook(HWND hOwner);
DLLEXPORT void RemoveHook();

//------------------------------------------------

BOOL WINAPI DllMain(HANDLE hinstDLL, DWORD dwReason, LPVOID lpvReserved)
{
switch (dwReason)
{
case DLL_THREAD_ATTACH:
case DLL_PROCESS_ATTACH:
g_hInstance = (HINSTANCE)hinstDLL;
break;
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
RemoveHook();
break;
}
return TRUE;
}

//------------------------------------------------

DLLEXPORT void InstallHook(HWND hOwner)
{
MyHwnd = hOwner;
MessageBox(MyHwnd, "InstallHook", "From DLL", MB_OK);
}

//------------------------------------------------

DLLEXPORT void RemoveHook()
{
MessageBox(MyHwnd, "RemoveHook", "From DLL", MB_OK);
}

//------------------------------------------------

If you sre static linking remeber to include the .lib file that is generated when you compile the .dll.

You should be able to work off this by throwing you hook code into the correct functions (InstallHook, RemoveHook).

Any more questions feel free to ask. :)
If the post helped please rate!

[Chazwazza]

No you see, I can call the functions in the DLL without hassle, and I can display message boxes etc. But the actual problem is with the hooks, not with the DLL. The DLL seems to be working, as I can add code to call a MessageBox in the InstallHook funciton, and it works. The problem is the hook not being set properly.

Thanks.

[UnfitElf]

Haha, ok, sorry i misunderstood your question. Its been a very long day and im proably not as sharp as i should be.

I do not have time to reply now, but i will post back 2moro after work.

[Chazwazza]

OK thanks, I look forward to hearing from you, it's hard to continue with writing this program when I can't work out how to get one of the major features to work. :P

[UnfitElf]

Ok, this is a working sample. :)

dll file
//------------------------------------------------

#include <windows.h>

//------------------------------------------------

#define DLLEXPORT __declspec(dllexport)

//------------------------------------------------

#pragma data_seg(".HOOKDATA")//Shared data among all instances.
HHOOK hook = NULL;
HWND hwnd = NULL;
#pragma data_seg()

//------------------------------------------------

#pragma comment(linker, "/SECTION:.HOOKDATA,RWS")

//------------------------------------------------

HINSTANCE hInst = NULL;

//------------------------------------------------

DLLEXPORT void InstallHook(HWND hOwner);
DLLEXPORT void RemoveHook();

LRESULT WINAPI HookProc(int nCode, WPARAM wParam, LPARAM lParam);

//------------------------------------------------

BOOL WINAPI DllMain(HANDLE hinstDLL, DWORD dwReason, LPVOID lpvReserved)
{
switch (dwReason)
{
case DLL_THREAD_ATTACH:
case DLL_PROCESS_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
hInst = (HINSTANCE)hinstDLL;
return TRUE;
}

//------------------------------------------------

DLLEXPORT void InstallHook(HWND hOwner)
{
hwnd = hOwner;

hook = SetWindowsHookEx(WH_KEYBOARD, HookProc, hInst, 0);

if (!hook)
{
MessageBox(hwnd, "Error setting hook", "Error", MB_OK);
}
}

//------------------------------------------------

DLLEXPORT void RemoveHook()
{
if (hook)
UnhookWindowsHookEx(hook);
}

//------------------------------------------------

LRESULT WINAPI HookProc(int nCode, WPARAM wParam, LPARAM lParam)
{
if(nCode >= 0)
{
if(lParam & (1 << 31))
{
SendMessage(hwnd, WM_USER+755, wParam, lParam);
}
}
return CallNextHookEx(hook, nCode, wParam, lParam);
}

//------------------------------------------------

.exe file
//---------------------------------------------------------------------------------

// DLL example. i.e. calling a function from a .dll

//---------------------------------------------------------------------------------

// main.cpp

//---------------------------------------------------------------------------------

#include <windows.h>

//---------------------------------------------------------------------------------

#define DLLIMPORT __declspec(dllimport)

DLLIMPORT void InstallHook(HWND hOwner);
DLLIMPORT void RemoveHook();

//---------------------------------------------------------------
// Globals

HINSTANCE hInst;
HWND HwndMe;
TCHAR *szTitle = "MyProject";
TCHAR *szWindowClass = "MyProject";

//---------------------------------------------------------------
// Windows setup

LRESULT CALLBACK WindowProcedure (HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam);

int WINAPI WinMain (HINSTANCE hThisInstance,
HINSTANCE hPrevInstance,
LPSTR lpszArgument,
int nFunsterStil)

{
//HWND hwnd;
MSG messages; // Here messages to the application are saved
WNDCLASSEX wincl; // Data structure for the windowclass

// The Window structure
wincl.hInstance = hThisInstance;
wincl.lpszClassName = szWindowClass;
wincl.lpfnWndProc = WindowProcedure; // This function is called by windows
wincl.style = CS_DBLCLKS; // Catch double-clicks
wincl.cbSize = sizeof (WNDCLASSEX);

// Use default icon and mouse-pointer
wincl.hIcon = LoadIcon (NULL, IDI_APPLICATION);
wincl.hIconSm = LoadIcon (NULL, IDI_APPLICATION);
wincl.hCursor = LoadCursor (NULL, IDC_ARROW);
wincl.lpszMenuName = NULL; // No menu
wincl.cbClsExtra = 0; // No extra bytes after the window class
wincl.cbWndExtra = 0; // structure or the window instance
// Use Windows's default color as the background of the window
wincl.hbrBackground = (HBRUSH) COLOR_BACKGROUND;

// Register the window class, and if it fails quit the program
if (!RegisterClassEx (&wincl))
return 0;

// The class is registered, let's create the program
HwndMe = CreateWindowEx (
0, // Extended possibilites for variation
szWindowClass, // Classname
szTitle, // Title Text
WS_OVERLAPPEDWINDOW, // default window
CW_USEDEFAULT, // Windows decides the position
CW_USEDEFAULT, // where the window ends up on the screen
300, // The programs width
300, // and height in pixels
HWND_DESKTOP, // The window is a child-window to desktop
NULL, // No menu
hThisInstance, // Program Instance handler
NULL // No Window Creation data
);

hInst = hThisInstance;

// Make the window visible on the screen
ShowWindow (HwndMe, nFunsterStil);

InstallHook(HwndMe);

// Run the message loop. It will run until GetMessage() returns 0
while (GetMessage (&messages, NULL, 0, 0))
{
// Translate virtual-key messages into character messages
TranslateMessage(&messages);
// Send message to WindowProcedure
DispatchMessage(&messages);
}
// The program return-value is 0 - The value that PostQuitMessage() gave
return (int)messages.wParam;
}

//------------------------------------------------------------------------------
// Windows proc

LRESULT CALLBACK WindowProcedure (HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
{
switch (message)
{
case WM_USER+755:
MessageBeep(MB_OK);
break;
case WM_CREATE:
break;
case WM_CLOSE:
case WM_ENDSESSION:
RemoveHook();
DestroyWindow(hwnd);
break;
case WM_DESTROY:
PostQuitMessage(0);
break;
default:
return DefWindowProc (hwnd, message, wParam, lParam);
}
return 0;
}

//------------------------------------------------------------------------------


Hope this helps. Thanks for asking the question, gave me a good excuse to muck around learning this dll stuff :D

[Chazwazza]

It works! Thank you so much, I can't even begin to express how much this helps. :) (I will start by rating your posts)

My project is extremely messy, I think this is a good time to rebuild it from the start. Thank you so very much.

[UnfitElf]

Most welcome, good luck :)