如何设置Numpy并行计算度以使用全部CPU核心？

Hey there! I see you're having trouble getting numpy to utilize all 8 cores on your machine—right now it's only using 4, even after following a tutorial meant to adjust thread limits. Let's break this down and get it sorted.

First, looking at your numpy.show_config() output, your numpy is built with MKL (Intel Math Kernel Library) as its backend. That's the key detail here: MKL controls its own thread count separately from generic numpy settings, which is why the tutorial you tried probably didn't work (it was likely written for OpenBLAS or another backend, not MKL).

Here's how to force MKL to use all 8 cores:

1. Temporary Fix (Current Python Session Only) #

You need to set the MKL-related environment variables before importing numpy—MKL initializes its thread count the moment numpy is first loaded, so setting these variables after importing numpy won't have any effect. Add this at the very top of your script:

import os
# Configure MKL and related libraries to use 8 threads
os.environ["MKL_NUM_THREADS"] = "8"
os.environ["NUMEXPR_NUM_THREADS"] = "8"
os.environ["OMP_NUM_THREADS"] = "8"

# Now import numpy
import numpy as np

2. Permanent Fix (Applies to All Sessions) #

If you want this setting to stick every time you run numpy, you have two straightforward options:

Option A: Conda Environment (Recommended for Anaconda/Miniconda) #

Edit your ~/.condarc file (create it if it doesn't exist) and add these lines:

env_vars:
  MKL_NUM_THREADS: "8"
  NUMEXPR_NUM_THREADS: "8"
  OMP_NUM_THREADS: "8"

Save the file, and the next time you launch a conda environment, these variables will automatically be active.

Option B: Shell Configuration (bash/zsh) #

Add these lines to your ~/.bashrc (for bash users) or ~/.zshrc (for zsh users):

export MKL_NUM_THREADS=8
export NUMEXPR_NUM_THREADS=8
export OMP_NUM_THREADS=8

Run source ~/.bashrc (or source ~/.zshrc) to apply changes immediately, or restart your terminal for the settings to take full effect.

3. Verify the Fix #

To confirm everything is working as expected:

• Run a large numpy.dot operation and check your system monitor (like htop or top)—you should see all 8 cores being utilized.
• Alternatively, you can check the MKL thread count directly in Python:

import numpy as np
# If you have the mkl module installed
from mkl import get_max_threads
print(f"MKL max threads: {get_max_threads()}")  # Should output 8

Why Your Original Tutorial Failed #

Most generic "adjust numpy threads" guides target OpenBLAS, which uses the OPENBLAS_NUM_THREADS environment variable. Since your setup uses MKL, that variable doesn't affect it—you need to use MKL_NUM_THREADS instead, which is what we've configured here.

内容的提问来源于stack exchange，提问作者Francesco Cariaggi