I’m revisiting my unfinished binary hook engine that I started back in 2007 to hopefully complete it and use it in some current projects.
For people new to such things. It’s a programming mechanism that allows you to inject/load a DLL (usually) into a target process at the same time, or after it’s loaded to allow you to modify, and, or, hijack parts of it’s code. See: Hooking at wikipedia
What game and otherwise hackers might use to make “hacks”, bots, etc. But that’s not all. It’s along the same lines as “plug-ins”, “ad-dons”, etc. For instance there are systems that use a hook system to add TeamSpeak to games so you can graphically see who is speaking. Firewalls and security software like real time anti-virus programs might use a hook system too.
When you read about hooking systems you will mostly see API hooking. But as the Wikipedia article covers to some degree is that the whole “hooking” concept is actually pretty broad topic. Often maligned do to it’s notorious uses in malware like rootkits. But there are as many of not more positive, helpful, and useful reasons to use hooking mechanisms.
Typically one will use a DLL as they make the most sense as the main if not the only official dynamic code module device in all Windows OSes. Although there are low level alternatives. One could make their own system to add code space inside of a process with out using an actual DLL.
Back on my own engine design..
It’s focused as a general Windows binary hooking system to not only hook APIs but additionally arbitrary code functions and an ability to “tap” into arbitrary points inside of functions for filtering and monitoring. Additionally have multilateral support for break-point/exception hooks, more transparent (not requiring code modification) HWBP (“Hardware Breakpoint”) and page guard type hooks.
Some existing related hooking systems of note:
1) Microsoft Detours
Fee for non-commercial use. You must pay to get native 64bit support.
It seems to be fairly popular. I’ve seen it’s use in some game bots and I’ve examined it in detail but have yet to try it for a project my self. As with a lot of Microsoft technologies I tend to avoid them for good reason.
2) EasyHook – The reinvention of Windows API Hooking
Appears to be free for non-commercial use.
Is full featured supporting native 64bit, .NET, etc., and with source code.
There is a lot of hype and self back-patting (although a lot has been removed) claiming “reinvention”, etc., but when looking at the low level details (the source) it’s not much different then any other, doing a lot of the same stuff using the same techniques that were “invented” years before with maybe the exception of things like 64bit support and his .NET interface.
3) EliCZ’s APIHooks
A low level hooking and systems level utility system. Supports 64bit hooking also, plus most if not all Windows OS’es starting with Windows 95.
Provides a lot of low level flexibility on how API and code hooks are done, with support for unhooking APIs et al. Used internally by nProtect GameGuard anti-cheat system.
EliCZ’s was nice enough to help me with my own load/inject DLL on process create method I use in my script/macro system MacroMonkey!
He’s one hell of a great systems programmer who really understands the lowest Windows system details.
My only criticism and the reason I think it has not been as popular as it could be is the lack of documentation, examples, and lack of a forum for feedback.
4) Madshi’s madCodeHook
This is my personal favorite that I have consistently used on many projects for years.
Has 64bit support, supports all Windows OS from 95 to Windows 7, with correct stable system wide hooking, etc. A lot usefully utility support (see: http://help.madshi.net/madCodeHook.htm).
I can attest that to what he says:
Using madCodeHook is very simple and straightforward. “It just works”
If you need the feature you can properly unhook APIs using it’s “safe unhooking” (keeps interlocked count). It’s very stable, well supported, documented, often updated, used in many commercial products, etc.
I recall it was used with with a lot of the great 3rd party tools for the MORPG Anarchy Online back when I used to play it.
Unfortunately there is no longer a free version of madCHook due to his trouble with idiots using it for malware, and the license fee has more then doubled since the last time I paid. Although if you want a stable, commercial quality, and perhaps the best API hooking system available, then it’s more then worth the cost.
There is many more I left out. Including all the free code, articles, and tutorials you can find on places like THE CODE PROJECT, GameDeception, etc.
The main impetus for making my own hook system is the general lack of very low level support and just some things none of these have yet.
AFAIK none properly support functions that have a C++ “this” (passed in register ECX) support. They don’t facilitate just taping into code at arbitrary points. They don’t consistently (at least not documented) save and restore register and flags contexts. No support for nondestructive HWBP, or page guard methods, etc.
Most mainly seem to be stuck in the same API hook “subclass” paradigm.
I’ve spend a lot of time tediously crafting manually custom little ASM proxy hook stubs that could be created automatically for me.
Plus my beloved madCHook is a little bulky with features and support I don’t need.
A lot of things other system level programmers and reversers think about.
For functions to hook some times maybe you just want subclass them but then run into problems trying to handle thiscall or other difficult calling conventions, and, or, want to work with processes complied with Profile Guided Optimization (with odd registers passed as arguments, and, or registers that can’t be altered).
For some type of live reverse engineering monitoring maybe you need to just tap into some code inside of a complex function et al.
And it’s preferred to have the least amount of overhead and the smallest footprint inside of your target.
These are the questions I’m trying answer and needs fulfilled with my own hooking system.