使用PyInstaller打包MXNet代码遇库加载错误及dynlib/dll疑问求助

Hey there, let's tackle this problem step by step. First, let's clear up what a dynlib/dll is, then walk through fixes for your MXNet packaging issue.

What's a dynlib/dll? #

• On Windows, DLL stands for Dynamic Link Library — these are shared files that programs load at runtime to access pre-written code (like MXNet's GPU/CPU computation logic).
• On Linux/macOS, dynlib (short for dynamic library) is the equivalent; your libmxnet.so is exactly this type of file for Linux.
• The core issue here is that your frozen PyInstaller executable can't locate or properly load this shared library when it runs.

Step-by-Step Fixes #

1. Check Dependencies of libmxnet.so #

First, make sure your libmxnet.so doesn't have missing system dependencies. Run this command in your terminal:

ldd /path/to/your/libmxnet.so

Look for any lines marked not found — these are missing libraries that MXNet needs to run. For example, if you're using a GPU-enabled MXNet, you might be missing CUDA libraries; for CPU-only, check for OpenBLAS or other linear algebra libraries.

• If there are missing dependencies, you'll need to either:
  • Install those libraries on the target machine where you'll run the executable, or
  • Add them to your PyInstaller spec file's binaries list so they get packaged alongside libmxnet.so.

2. Fix Your PyInstaller Spec File #

Your spec file needs to correctly include both the MXNet Python modules and the libmxnet.so library. Here's a corrected example spec file tailored to your setup:

a = Analysis(
    ['your_main_script.py'],  # Replace with your actual main script name
    pathex=['/home/rit/test/exe'],  # Path to your project root
    binaries=[('/path/to/your/libmxnet.so', '.')],  # Package libmxnet.so into the output root
    datas=[('/path/to/your/python3.5/dist-packages/mxnet', 'mxnet')],  # Package the full MXNet Python module
    hiddenimports=['mxnet', 'mxnet.libinfo', 'mxnet.ndarray', 'mxnet.symbol'],  # Force PyInstaller to detect these imports
    hookspath=[],
    runtime_hooks=[],
    excludes=[],
)
pyz = PYZ(a.pure, a.zipped_data)
exe = EXE(
    pyz,
    a.scripts,
    a.binaries,
    a.zipfiles,
    a.datas,
    name='test',
    console=True,  # Keep this if you need terminal output
    upx=True,  # Optional: Compress the executable
)

Key notes here:

• binaries: Ensures libmxnet.so is placed directly in the same folder as your executable.
• datas: Copies the entire MXNet Python package into the frozen app's structure.
• hiddenimports: PyInstaller sometimes misses MXNet's internal modules, so we explicitly list them here.

3. Set Runtime Library Path #

Add this snippet at the very top of your main Python script to ensure the executable looks for libmxnet.so in its own directory first:

import os
import sys

if getattr(sys, 'frozen', False):
    # We're running the frozen executable, not raw Python
    app_dir = os.path.dirname(sys.executable)
    # Update LD_LIBRARY_PATH to prioritize the app's directory
    os.environ['LD_LIBRARY_PATH'] = f"{app_dir}:{os.environ.get('LD_LIBRARY_PATH', '')}"

This tells Linux to look for shared libraries in the same folder as your executable before checking system paths.

4. Verify Version Consistency #

Make sure:

• The mxnet folder you copied into your project matches the version of libmxnet.so you're using. Mixing different versions (e.g., a Python 3.5 MXNet module with a newer libmxnet.so) will cause loading failures.
• You've uninstalled any pip-installed MXNet versions to avoid conflicts during packaging.

5. Test the Packaged App #

After rebuilding with the fixed spec file, navigate to the dist/test directory and run your executable directly from there. This ensures it can access all packaged files without path issues.

内容的提问来源于stack exchange，提问作者Ritesh Dubal