Inline Hook目标程序函数后程序崩溃的原因排查

目标程序代码 #

#include <iostream>

int sum(int x, int y) {
    std::cout << "function";
    return x + y;
}

int main()
{
    while (true) {
        std::cin.get();
        std::cout << sum(1, 2);
    }
}

Hook实现代码 #

// dllmain.cpp : Define o ponto de entrada para o aplicativo DLL.
#include "pch.h"
#include "Windows.h"
#include <iostream>

typedef int(__cdecl* sum) (int x, int y);

sum osum;

int __cdecl hsum(int x, int y) {
    return osum(x, y*5);
}

bool Detour32(void* src, void* dst, int len)
{
    if (len < 5) return false;

    DWORD curProtection;
    VirtualProtect(src, len, PAGE_EXECUTE_READWRITE, &curProtection);

    memset(src, 0x90, len);

    uintptr_t relativeAddress = ((uintptr_t)dst - (uintptr_t)src) - 5;

    *(BYTE*)src = 0xE9;
    *(uintptr_t*)((uintptr_t)src + 1) = relativeAddress;

    DWORD temp;
    VirtualProtect(src, len, curProtection, &temp);

    return true;
}

char* TrampHook32(BYTE* src, BYTE* dst, const intptr_t len)
{
    // Make sure the length is greater than 5
    if (len < 5) return 0;

    // Create the gateway (len + 5 for the overwritten bytes + the jmp)
    BYTE* gateway = (BYTE*)VirtualAlloc(0, len + 5, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);

    //Write the stolen bytes into the gateway
    memcpy(gateway, src, len);

    // Get the gateway to destination addy
    intptr_t  gatewayRelativeAddr = ((intptr_t)src - (intptr_t)gateway) - 5;

    // Add the jmp opcode to the end of the gateway
    *(char*)((intptr_t)gateway + len) = 0xE9;

    // Add the address to the jmp
    *(intptr_t*)((intptr_t)gateway + len + 1) = gatewayRelativeAddr;

    // Perform the detour
    Detour32(src, dst, len);

    return (char*)gateway;
}

DWORD WINAPI HackThread(HMODULE hModule) {
    //Create Console
    AllocConsole();
    FILE* f;
    freopen_s(&f, "CONOUT$", "w", stdout);

    uintptr_t moduleBase = (uintptr_t)GetModuleHandle(L"testtt.exe");
    osum = (sum)(moduleBase + 0x123d0);
    osum = (sum)TrampHook32((BYTE*)osum, (BYTE*)hsum, 5);

    fclose(f);
    FreeConsole();
    return 0;
}
BOOL APIENTRY DllMain(HMODULE hModule,
    DWORD  ul_reason_for_call,
    LPVOID lpReserved
)
{
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        CloseHandle(CreateThread(nullptr, 0, (LPTHREAD_START_ROUTINE)HackThread, hModule, 0, nullptr));
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}

问题说明 #

已确认DLL注入逻辑正常，Cheat Engine调试显示跳转生效，程序进入自定义替换函数后执行部分指令触发崩溃，需定位崩溃原因。

1. 指令截断（最高概率） #

当前调用TrampHook32时硬编码偷取原函数前5字节，如果5字节没有刚好覆盖完整的多条指令，而是截断了某一条指令，网关执行半条指令时会触发非法指令异常直接崩溃。
解决方案：到Ghidra中查看sum函数入口的汇编指令，统计前N条完整指令的总长度，只要总长度≥5，就将TrampHook32的第三个参数改为这个总长度，禁止硬编码为5。

2. 调用约定不匹配 #

当前函数指针使用__cdecl约定，如果原sum函数编译时采用__stdcall、__fastcall等其他约定，会导致栈平衡错误，栈结构混乱触发崩溃。
解决方案：通过原函数汇编结尾判断约定，结尾为ret是__cdecl，结尾为ret 8（两个int参数占8字节）是__stdcall，对应修改函数指针的调用约定即可。

3. 架构不匹配 #

该Hook是32位专用实现，若DLL编译为64位，或目标程序为64位，架构不一致必然触发崩溃。需确认目标程序和DLL均为32位架构。

4. 函数入口地址错误 #

硬编码的偏移0x123d0可能不是sum函数的正确入口，可在Cheat Engine中跳转到moduleBase + 0x123d0地址，对比汇编是否与Ghidra中sum函数开头汇编完全一致，确认地址正确性。

5. 控制台操作异常 #

Hook完成后立即执行fclose和FreeConsole可能带来额外运行时异常，可先注释这两行代码，等Hook逻辑稳定后再调试控制台相关逻辑。

内容的提问来源于stack exchange，提问作者felipebubu