如何在不可修改的C++源码中调用DLL重载的LoadTrajectory函数？

Absolutely, you can get the main program to call your overridden LoadTrajectory function—even with no access to modify its source code. The solution hinges on vtable hooking (virtual function table manipulation), a practical technique for intercepting virtual function calls in C++ when you can’t alter the caller code. Here’s how to pull it off:

Core Idea #

C++ uses a shared virtual function table (vtable) for every class with virtual functions. Every instance of TrajectoryLoader holds a pointer to this vtable. By replacing the entry for LoadTrajectory in the original class’s vtable with your overridden function’s address, all existing and future TrajectoryLoader instances will automatically call your code instead of the original.

Step-by-Step Implementation #

1. Define Your Subclass #

First, finalize your subclass in the DLL as you already started, overriding LoadTrajectory with your custom logic:

#include "TrajectoryLoader.h" // Third-party header
#include <string>

class MyTrajectoryLoader : public TrajectoryLoader {
public:
    bool LoadTrajectory(const std::string& filePath) override {
        // Handle your custom trajectory formats first
        if (isCustomFormat(filePath)) {
            // Add your custom loading logic here
            return true;
        }
        // Fall back to the original implementation if needed
        return TrajectoryLoader::LoadTrajectory(filePath);
    }

private:
    bool isCustomFormat(const std::string& path) {
        // Add logic to detect your supported formats (e.g., file extensions)
        return path.ends_with(".myfmt");
    }
};

2. Locate the LoadTrajectory Entry in the VTable #

You need to find the index of LoadTrajectory in the TrajectoryLoader vtable. Here’s how to do it safely:

• Create a small test program that prints the addresses of all virtual functions in TrajectoryLoader and your subclass. Compare the addresses to pinpoint which index maps to LoadTrajectory.
• Alternatively, use debug tools like Visual Studio’s debugger or dumpbin /headers to inspect the class’s vtable layout directly.

For example, if LoadTrajectory is the second virtual function (0-indexed), the index would be 1.

3. Hook the VTable in Your DLL’s Entry Point #

In your DLL’s DllMain function, modify the original vtable to redirect LoadTrajectory calls to your implementation:

#include <windows.h>
#include <memory>

// Replace this with the actual index you verified
const size_t LOAD_TRAJECTORY_VTABLE_INDEX = 1;

BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved) {
    switch (ul_reason_for_call) {
    case DLL_PROCESS_ATTACH:
        // Create a temporary instance to access the original vtable
        std::shared_ptr<TrajectoryLoader> tempOriginal = std::make_shared<TrajectoryLoader>();
        void** originalVtable = *(void***)(tempOriginal.get());

        // Create an instance of your subclass to get your function's address
        std::shared_ptr<MyTrajectoryLoader> tempCustom = std::make_shared<MyTrajectoryLoader>();
        void** customVtable = *(void***)(tempCustom.get());
        void* customLoadFunc = customVtable[LOAD_TRAJECTORY_VTABLE_INDEX];

        // Make the vtable entry writable (vtables are usually read-only by default)
        DWORD oldProtection;
        VirtualProtect(&originalVtable[LOAD_TRAJECTORY_VTABLE_INDEX], sizeof(void*), PAGE_EXECUTE_READWRITE, &oldProtection);

        // Replace the original function pointer with your custom one
        originalVtable[LOAD_TRAJECTORY_VTABLE_INDEX] = customLoadFunc;

        // Restore the original memory protection to avoid unexpected behavior
        VirtualProtect(&originalVtable[LOAD_TRAJECTORY_VTABLE_INDEX], sizeof(void*), oldProtection, &oldProtection);
        break;

    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}

Critical Notes #

• VTable Index Accuracy: If you use the wrong index, the program will crash immediately. Always double-check the index with a test build or debugger.
• Thread Safety: If the main program is multi-threaded, wrap the vtable modification in a critical section (e.g., CriticalSection on Windows) to prevent race conditions.
• Version Compatibility: If the third-party library updates and changes the order of virtual functions, your hook will break. Re-verify the vtable index after any library version update.
• Memory Protection: Always restore the original memory protection after modifying the vtable to maintain stability.

Alternative (If Applicable) #

If the main program dynamically links to the third-party library, you could create a "proxy" DLL with identical exported symbols, redirecting calls to your implementation where needed. However, this requires replacing the original library file, which may not align with your goal of avoiding modifications to the original setup.

内容的提问来源于stack exchange，提问作者Mortimer Pignon