如何在远程服务器重建本地Python Conda环境并解决依赖缺失问题

Got it, let's walk through how to solve this—this is a super common headache when moving Python environments between local machines and remote servers, especially with packages that rely on system-level libraries like dlib or boost. Here are the most reliable solutions:

1. Use Conda to Manage All Possible Low-Level Dependencies (Best Practice) #

The root issue here is that pipreqs only tracks Python-level imports, not the system libraries or conda-managed dependencies needed for packages like dlib. Instead, lean into conda's strength at handling binary dependencies:

• Reinstall problematic packages via conda locally: If you originally installed dlib/boost via pip, uninstall them first:

  pip uninstall -y dlib boost-python
  

  Then install them from conda-forge (which has pre-built binaries that include all required low-level dependencies):

  conda install -c conda-forge dlib boost-cpp
  

• Export a complete environment.yml: Instead of using --from-history (which only includes packages you explicitly installed), export the full environment to capture all transitive dependencies:

  conda env export > environment.yml
  

  Then open the file and delete the prefix: /path/to/your/local/env line—this path won't exist on the remote server.
• Recreate the environment remotely: On your server, run:

  conda env create -f environment.yml
  

  Conda will handle installing all dependencies, including the underlying boost libraries needed for dlib, without needing manual system-level installs.

2. Combine Conda + Pip with Pre-Installed System Dependencies #

If some packages must be installed via pip (e.g., no conda package exists), you'll need to set up the remote server's system libraries first:

• Install system-level dependencies on the remote server:
  • For Ubuntu/Debian-based systems:

    sudo apt-get update && sudo apt-get install -y build-essential cmake libboost-all-dev libopencv-dev
    

  • For CentOS/RHEL-based systems:

    sudo yum install -y gcc-c++ cmake boost-devel opencv-devel
    

• Update your environment.yml to include pip packages: Add a pip section to your yml file so conda handles pip installs after setting up the conda environment:

  name: your_env_name
  channels:
    - conda-forge
    - defaults
  dependencies:
    - python=3.9
    - numpy>=1.22
    # Add other conda-managed packages here
    - pip:
      - dlib==19.24.2
      # Add other pip-only packages here
  

• Recreate the environment remotely: Run the same conda env create -f environment.yml command—conda will set up the base environment, then pip will install your packages using the pre-installed system libraries.

3. Workaround for Servers Without Sudo Access #

If you can't install system packages on the remote server, use conda to install the required low-level libraries in your user environment:

• Install system dependencies via conda locally:

  conda install -c conda-forge cmake boost-cpp opencv
  

• Export the environment.yml as before (remove the prefix line).
• Recreate remotely: When you run conda env create -f environment.yml, conda will install these system-level libraries into your conda environment. Then you can safely install pip packages like dlib, as they'll use the conda-provided libraries instead of system ones.

Key Tips to Avoid Headaches #

• Match OS versions if possible: If your local machine is Ubuntu and the remote is CentOS, some conda packages might have compatibility issues. Stick to conda-forge packages, which are more cross-platform.
• Relax version constraints: If you run into dependency conflicts, try changing exact versions (like numpy=1.22) to minimum versions (like numpy>=1.22) to let conda resolve conflicts automatically.
• Test incrementally: If the full environment fails to build, try installing packages one by one to isolate which dependency is causing the issue.

内容的提问来源于stack exchange，提问作者Harsh Gupta