如何用Cython在独立进程运行函数（不与Python交互）并复现多进程行为

Alright, let's tackle your two questions head-on—since you want to avoid any Python interpreter interaction in the child processes, we'll be using pure C-level process APIs instead of Python's multiprocessing module (which relies on the Python runtime under the hood).

1. Running a Function in a Separate Process with Cython (No Python Interaction) #

The core idea here is to compile your Cython code to pure C, then use OS-native process creation functions (like fork() on POSIX systems, or CreateProcess() on Windows) to spawn child processes that run your C-level functions directly.

Step 1: Cython Code (process_demo.pyx) #

This example uses POSIX APIs (works on Linux/macOS; we'll note Windows adjustments below):

# distutils: language = c
# distutils: extra_compile_args = -Wall -O2

from posix cimport fork, sleep
from stdlib cimport exit

# Pure C-level function (no Python objects, no GIL)
cdef void worker(int delay) nogil:
    while True:
        # Replace with your actual C-level work here
        sleep(delay)

def spawn_worker(int delay):
    cdef pid_t pid
    pid = fork()
    if pid == 0:
        # Child process: run the worker and never return to Python
        worker(delay)
        exit(0)  # Just in case the loop breaks
    elif pid < 0:
        raise OSError("Failed to fork child process")
    else:
        print(f"Spawned child process with PID {pid}")

Step 2: Compilation Script (setup.py) #

To turn this into a compiled extension:

from setuptools import setup
from Cython.Build import cythonize

setup(
    name="cython_process_demo",
    ext_modules=cythonize("process_demo.pyx"),
)

Step 3: Build and Run #

Compile with:

python setup.py build_ext --inplace

Then use it in Python (the parent process uses Python, but the child runs pure C):

from process_demo import spawn_worker
spawn_worker(1)
# The parent can exit or do other work; the child keeps running

Windows Note: Replace fork() with Windows' CreateProcessA()—you'll need to cimport Windows headers and adjust the process creation logic to run a C function (you may need to wrap the worker in a standalone entry point).

2. Replicating the Python Multiprocessing Example with Cython cdef Functions #

Let's map your original Python code directly to pure C-level processes using Cython. We'll create two separate cdef workers and spawn them as independent child processes.

Full Cython Code (multi_process_demo.pyx) #

# distutils: language = c
# distutils: extra_compile_args = -Wall -O2

from posix cimport fork, sleep
from stdlib cimport exit

# Pure C-level worker 1
cdef void func1(int n) nogil:
    while True:
        # Your custom C-level work here (no Python code!)
        sleep(n)

# Pure C-level worker 2
cdef void func2(int n) nogil:
    while True:
        # Your other custom C-level work here
        sleep(n)

def spawn_workers():
    cdef pid_t pid1, pid2

    # Spawn first worker
    pid1 = fork()
    if pid1 == 0:
        func1(1)
        exit(0)
    elif pid1 < 0:
        raise OSError("Failed to spawn func1 process")

    # Spawn second worker
    pid2 = fork()
    if pid2 == 0:
        func2(1)
        exit(0)
    elif pid2 < 0:
        raise OSError("Failed to spawn func2 process")

    print(f"Spawned func1 (PID: {pid1}) and func2 (PID: {pid2})")

Compilation and Usage #

Use the same setup.py as before, compile with python setup.py build_ext --inplace, then run:

from multi_process_demo import spawn_workers
spawn_workers()
# Parent process can exit; both child processes run indefinitely

Key Points to Note #

• All worker functions are cdef void ... nogil: this ensures they're compiled to pure C, no Python objects, and don't hold the Global Interpreter Lock.
• We use OS-native fork() instead of Python's multiprocessing.Process—this means the child processes have no Python runtime attached (they run pure C code).
• For Windows, you'll need to adjust the process creation logic since fork() isn't available—use CreateProcess() and a standalone C entry point for each worker.

内容的提问来源于stack exchange，提问作者Greg