如何从multiprocessing Pool工作进程获取实际核心编号？

Hey there! Let's break down what's happening here and fix that core number issue for you.

First off, the reason you're seeing numbers like 3-12 (or higher) is that you're almost certainly not grabbing actual CPU core IDs—you're probably picking up process IDs (PIDs) instead. System-assigned PIDs are globally incrementing and have nothing to do with your CPU core count, so they'll always exceed 10. Even if you were capturing core-related values, most systems use 0-based indexing for 10 cores (0-9), so you'd need to shift that to get 1-10.

Here are two solid solutions to get the core numbering you want:

Solution 1: Dynamically Get the Current Running Core (Simple, No Binding) #

This method grabs the core a task is currently running on, then converts it to your desired 1-10 range. It uses the psutil library (install first with pip install psutil):

import psutil
from multiprocessing import Pool
import logging

logger = logging.getLogger(__name__)

def _initializer(config, cfg):
    # Keep your existing initialization logic here
    pass

def your_task(args):
    # Get the 0-based core ID the process is currently using (0-9 for 10 cores)
    current_core_0based = psutil.Process().cpu_num()
    # Convert to 1-10 numbering
    core_number = current_core_0based + 1
    logger.info(f"Task running on core {core_number}")
    # Add your task logic here...

if __name__ == "__main__":
    # Explicitly set Pool size to 10 to match your 10-core machine
    with Pool(processes=10, initializer=_initializer, initargs=(config, cfg)) as p:
        # Submit your tasks (replace your_task_args with your actual arguments)
        p.map(your_task, your_task_args)

If you don't want to use a third-party library, os.sched_getaffinity(0) will give you the set of cores the process can run on—but it won't tell you the exact core it's using right now. For a stable, fixed ID, Solution 2 is better.

Solution 2: Assign Fixed 1-10 IDs to Each Pool Process (Core-Bound, Stable Numbers) #

If you want each process to map to a consistent core number (no changes from system scheduling), you can assign unique 1-10 IDs during initialization, and even bind each process to its corresponding core:

from multiprocessing import Pool, Value, Lock
import os
import logging

logger = logging.getLogger(__name__)

# Shared variable to track the next core ID to assign (starts at 1)
next_core_id = Value('i', 1)
# Lock to prevent race conditions when assigning IDs
core_id_lock = Lock()

# Global variable: each child process will hold its own unique core ID
core_id = None

def _initializer(config, cfg):
    global core_id
    # Safely assign a unique core ID
    with core_id_lock:
        core_id = next_core_id.value
        next_core_id.value += 1
    # Optional: Bind the process to its corresponding core (convert to 0-based)
    os.sched_setaffinity(0, {core_id - 1})
    # Keep your existing initialization logic here
    pass

def your_task(args):
    logger.info(f"Task running on core {core_id}")
    # Add your task logic here...

if __name__ == "__main__":
    with Pool(processes=10, initializer=_initializer, initargs=(config, cfg)) as p:
        p.map(your_task, your_task_args)

Key Notes: #

• Always explicitly set processes=10 to match your 10-core machine (avoids surprises with default Pool sizing).
• Core binding in Solution 2 is optional, but it guarantees your process won't jump between cores, keeping the ID completely stable.
• Always wrap your Pool startup in if __name__ == "__main__"—this is required for Windows and avoids weird issues on Unix systems.

That should get you the clean 1-10 core numbering you're looking for!

内容的提问来源于stack exchange，提问作者Jit