Python脚本Ubuntu单核心运行问题求助（依赖asammdf、numpy等）

Let’s break down the core of your problem first: your script relies on underlying libraries (specifically numpy/OpenBLAS) to handle multi-threaded processing, which works smoothly on Windows but only uses a single core on Linux. The key clue lies in the differences between your Windows and Linux numpy.show_config() outputs.

Key Observations from Numpy Configurations #

• On Windows, your OpenBLAS is built with language = f77
• On Linux, your OpenBLAS uses language = c and is installed in /usr/local/lib

This suggests your Linux OpenBLAS build might not have multi-threading enabled by default, or numpy isn’t leveraging it correctly.

Step-by-Step Solutions #

1. Force OpenBLAS to Use Multi-Threading via Environment Variables #

Linux builds of OpenBLAS often respect environment variables to control thread count. Try setting these before running your script:

export OPENBLAS_NUM_THREADS=$(nproc)  # Uses all available CPU cores
export OMP_NUM_THREADS=$(nproc)       # Fallback if OpenBLAS relies on OpenMP
python your_script.py

You can also test with a specific thread count (e.g., OPENBLAS_NUM_THREADS=4) to see if core utilization improves.

2. Verify Your OpenBLAS Build Supports Multi-Threading #

Check your current OpenBLAS configuration to confirm it’s compiled with multi-threading support:

openblas-config --show-config

Look for flags like USE_OPENMP=1 or NUM_THREADS=<number> in the output. If these are missing, you’ll need to recompile OpenBLAS with multi-threading enabled.

3. Recompile OpenBLAS with Multi-Threading #

If your existing OpenBLAS doesn’t support multi-threading, rebuild it from source:

# Install required build dependencies
sudo apt-get install gcc gfortran make

# Clone the OpenBLAS repository
git clone https://github.com/xianyi/OpenBLAS.git
cd OpenBLAS

# Compile with OpenMP for multi-threading and install
make USE_OPENMP=1
sudo make install

# Update system library paths
sudo ldconfig

4. Reinstall Numpy to Link Against the New OpenBLAS #

After rebuilding OpenBLAS, reinstall numpy to ensure it links against the multi-threaded version:

# Uninstall existing numpy
pip uninstall -y numpy

# Install numpy from source, forcing it to use the updated OpenBLAS
pip install numpy --no-binary :all:

Verify the new configuration with numpy.show_config() to confirm it’s using the updated OpenBLAS build.

5. Fix Python Compilation Dependencies (Ubuntu 16.04) #

Since you compiled Python 3.7.0 manually on Ubuntu 16.04, ensure you installed all required build dependencies before compiling Python:

sudo apt-get install build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python-openssl git

Missing dependencies can cause numpy to fail linking properly to system libraries like OpenBLAS.

Why Your Previous Fixes Didn’t Work #

Tools like os.sched_setaffinity or taskset only control which CPU cores a process can run on—they don’t force the underlying libraries to spawn multiple threads. If OpenBLAS isn’t configured for multi-threading, these tools won’t improve core utilization.

内容的提问来源于stack exchange，提问作者white91wolf