在C++嵌入Python3.7环境中为新建线程状态获取GIL的方法及代码问题排查

Let's break down what's going wrong in your code and fix the GIL handling so your Python scripts can run without deadlocks when importing libraries like numpy or pandas.

Key Issues in Your Current Code #

1. Misplaced GIL Release: You call PyEval_SaveThread() right at the start of executescript(), which releases the GIL held by the main thread's globalthread. This leaves your code in a state where no thread holds the GIL, and your subsequent attempts to switch thread states don't properly re-acquire it.
2. Flawed Thread State Switch Logic: After releasing the GIL, PyGILState_Check() will always return 0, leading you to call PyEval_RestoreThread(ts)—but this doesn't correctly associate the new thread state with the current thread while acquiring the GIL in this context.
3. Failed State Activation: The fact that PyGILState_GetThisThreadState() returns globalthread confirms your new thread state wasn't properly activated. Running Python code without the GIL causes deadlocks in C-extensions like numpy, which depend on it for thread safety.

Correct GIL & Thread State Management #

When creating a new PyThreadState to execute Python code (even in the same thread), follow these steps to ensure proper GIL acquisition:

1. Create the new thread state linked to your existing interpreter.
2. Acquire the GIL and set the new state as the active one for the current thread.
3. Execute your Python code safely now that you hold the GIL.
4. Clean up the thread state, release the GIL, and restore the original state.

Fixed Code Example #

Here's the revised executescript() function with adjustments to main():

#include <python.h>
#include <string>

PyThreadState* globalthread;

void executescript() {
    // Create a new thread state tied to our existing interpreter
    PyThreadState* ts = PyThreadState_New(globalthread->interp);
    
    // Acquire the GIL and set the new thread state as current
    PyEval_RestoreThread(ts);

    // Verify GIL ownership and correct thread state (optional, for debugging)
    int gilstate = PyGILState_Check(); // Should return 1 now
    PyThreadState* current_ts = PyGILState_GetThisThreadState(); // Should equal ts

    // Your Python script (using raw string literals for cleaner formatting)
    std::string str = R"(
def script():
    import sys
    sys.path.append('C:\\Python\\Python37\\Lib\\site-packages')
    print(sys.path)
    import numpy
    arr = numpy.array([1, 2, 3, 4, 5])
    print(arr)
script()
)";
    PyRun_SimpleString(str.c_str());

    // Release the GIL and save the current thread state
    PyThreadState* saved_ts = PyEval_SaveThread();
    
    // Clean up the new thread state
    PyThreadState_Clear(ts);
    PyThreadState_Delete(ts);

    // Restore the original global thread state and re-acquire the GIL
    PyEval_RestoreThread(globalthread);
}

int main() {
    Py_Initialize();
    PyEval_InitThreads();
    
    // Store the main thread's state (holds the GIL initially)
    globalthread = PyThreadState_Get();

    executescript();

    // Finalize clean-up
    PyThreadState_Swap(globalthread);
    Py_FinalizeEx();
    return 0;
}

Additional Tips for Multi-Threaded Scenarios #

If you plan to run executescript() in a separate C++ thread (not the main thread), use PyGILState_Ensure() instead of manual thread state management—it's simpler and less error-prone:

void executescript_in_separate_thread() {
    // Automatically acquire GIL and create a thread state if needed
    PyGILState_STATE gstate = PyGILState_Ensure();
    
    // Execute your Python code here
    
    // Release the GIL and clean up the thread state
    PyGILState_Release(gstate);
}

内容的提问来源于stack exchange，提问作者Revanth