Azure Databricks中Pyomo求解器IPOPT安装失败问题求助

问题描述 #

已在Azure Databricks上通过以下命令成功安装Pyomo：

pip install 'pyomo[optional]'

安装成功输出：

Successfully installed algopy-0.5.7 casadi-3.5.5 dill-0.3.5.1 et-xmlfile-1.1.0 mpmath-1.2.1 networkx-2.8.5 numdifftools-0.9.40 openpyxl-3.0.10 pint-0.19.2 ply-3.11 pyomo-6.4.1 python-louvain-0.16 pyyaml-6.0 sympy-1.10.1 xlrd-2.0.1 z3-solver-4.10.2.0

但尝试安装IPOPT求解器时出现两类错误：

错误1：找不到pkg-config工具 #

Collecting ipopt
  Downloading ipopt-1.0.3.tar.gz (42 kB)
     |████████████████████████████████| 42 kB 1.1 MB/s  eta 0:00:01
    ERROR: Command errored out with exit status 1:
     command: /databricks/python3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_97_jyim/ipopt_5ccc28f94eb5408181b160df9c422798/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_97_jyim/ipopt_5ccc28f94eb5408181b160df9c422798/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-zpxip7hb
         cwd: /tmp/pip-install-_97_jyim/ipopt_5ccc28f94eb5408181b160df9c422798/
    Complete output (13 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-_97_jyim/ipopt_5ccc28f94eb5408181b160df9c422798/setup.py", line 194, in <module>
        ext_module_data = handle_ext_modules_general_os()
      File "/tmp/pip-install-_97_jyim/ipopt_5ccc28f94eb5408181b160df9c422798/setup.py", line 173, in handle_ext_modules_general_os
        **pkgconfig("ipopt"))
      File "/tmp/pip-install-_97_jyim/ipopt_5ccc28f94eb5408181b160df9c422798/setup.py", line 100, in pkgconfig
        output = sp.Popen(["pkg-config", "--libs", "--cflags"] + list(packages),
      File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'pkg-config'

错误2：缺少IPOPT头文件IpStdCInterface.h #

Downloading ipopt-0.1.3.zip (109 kB)
     |████████████████████████████████| 109 kB 56.6 MB/s eta 0:00:01
Building wheels for collected packages: ipopt
  Building wheel for ipopt (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /databricks/python3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_97_jyim/ipopt_8a62a84a8dcf4bc5a832cb6bb3713b52/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_97_jyim/ipopt_8a62a84a8dcf4bc5a832cb6bb3713b52/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-8pgnfwqa
       cwd: /tmp/pip-install-_97_jyim/ipopt_8a62a84a8dcf4bc5a832cb6bb3713b52/
  Complete output (25 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/ipopt
  copying ipopt/__init__.py -> build/lib.linux-x86_64-3.8/ipopt
  running build_ext
  skipping 'src/cyipopt.c' Cython extension (up-to-date)
  building 'ipopt.cyipopt' extension
  creating build/temp.linux-x86_64-3.8
  creating build/temp.linux-x86_64-3.8/src
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/amitibo/code/Ipopt-3.10.1/include/coin -I/databricks/python3/lib/python3.8/site-packages/numpy/core/include -I/databricks/python3/include -I/usr/include/python3.8 -c src/cyipopt.c -o build/temp.linux-x86_64-3.8/src/cyipopt.o
  In file included from /databricks/python3/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
                   from /databricks/python3/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                   from /databricks/python3/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                   from src/cyipopt.c:237:
  /databricks/python3/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     17 | #warning "Using deprecated NumPy API, disable it with " \
        |  ^~~~~~~
  src/cyipopt.c:239:10: fatal error: IpStdCInterface.h: No such file or directory
    239 | #include "IpStdCInterface.h"
        |          ^~~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

解决办法 #

在Azure Databricks环境中，直接pip install ipopt会因系统依赖缺失失败，推荐以下几种可行方案：

方案1：通过conda安装预编译包（集群支持conda时优先使用） #

如果你的Databricks集群配置了conda环境，执行以下命令：

conda install -c conda-forge ipopt

方案2：安装系统依赖后编译安装 #

1. 先安装pkg-config和IPOPT系统库（需sudo权限，确保集群允许执行系统命令）：

   sudo apt-get update && sudo apt-get install -y pkg-config coinor-libipopt-dev
   

2. 再执行pip安装：

   pip install ipopt
   

方案3：使用Pyomo自带的求解器安装工具 #

Pyomo提供了内置工具自动处理求解器安装，执行：

python -m pyomo.scripting.pyomo_install --solver ipopt

内容的提问来源于stack exchange，提问作者user22