Python3.10与3.11下pip安装python-javabridge失败原因及修复

我尝试安装python-javabridge以搭建Java虚拟机，用于托管Python可调用的不同文件格式库。但该项目已停止维护，不支持Python>3.10版本。鉴于其重要性，我希望修复它以兼容Python3.11及更高版本。

我分别在Python3.10.14和3.11.8环境测试安装：

• Python3.10下执行以下命令，安装无问题：

> mamba create -n javabridge-test python==3.10.14
> conda activate javabridge-test  # 切换至目标环境
> pip install python-javabridge

> # 安装顺利完成

• 但在Python3.11.8环境（已提前安装numpy依赖）执行相同安装命令时，出现编译错误，核心报错为：_javabridge.c:196:12: fatal error: longintrepr.h: No such file or directory，完整错误信息如下：

Collecting python-javabridge
  Using cached python-javabridge-4.0.3.tar.gz (1.3 MB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy>=1.20.1 in /path/to/envs/javabridge-install/lib/python3.11/site-packages (from python-javabridge) (1.26.4)
Building wheels for collected packages: python-javabridge
  Building wheel for python-javabridge (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [46 lines of output]
      /path/to/envs/javabridge-install/lib/python3.11/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!
      
              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************
      
      !!
        dist.fetch_build_eggs(dist.setup_requires)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/javabridge
      copying javabridge/__init__.py -> build/lib.linux-x86_64-cpython-311/javabridge
      copying javabridge/jutil.py -> build/lib.linux-x86_64-cpython-311/javabridge
      copying javabridge/locate.py -> build/lib.linux-x86_64-cpython-311/javabridge
      copying javabridge/noseplugin.py -> build/lib.linux-x86_64-cpython-311/javabridge
      copying javabridge/wrappers.py -> build/lib.linux-x86_64-cpython-311/javabridge
      creating build/lib.linux-x86_64-cpython-311/javabridge/tests
      copying javabridge/tests/__init__.py -> build/lib.linux-x86_64-cpython-311/javabridge/tests
      copying javabridge/tests/test_cpython.py -> build/lib.linux-x86_64-cpython-311/javabridge/tests
      copying javabridge/tests/test_javabridge.py -> build/lib.linux-x86_64-cpython-311/javabridge/tests
      copying javabridge/tests/test_jutil.py -> build/lib.linux-x86_64-cpython-311/javabridge/tests
      copying javabridge/tests/test_wrappers.py -> build/lib.linux-x86_64-cpython-311/javabridge/tests
      creating build/lib.linux-x86_64-cpython-311/javabridge/jars
      copying javabridge/jars/cpython.jar -> build/lib.linux-x86_64-cpython-311/javabridge/jars
      copying javabridge/jars/rhino-1.7R4.jar -> build/lib.linux-x86_64-cpython-311/javabridge/jars
      copying javabridge/jars/runnablequeue.jar -> build/lib.linux-x86_64-cpython-311/javabridge/jars
      copying javabridge/jars/test.jar -> build/lib.linux-x86_64-cpython-311/javabridge/jars
      running build_ext
      javac -source 8 -target 8 /tmp/pip-install-gv_sm6fr/python-javabridge_ea4288a3fc0b4e16815944ed67e58092/java/org/cellprofiler/runnablequeue/RunnableQueue.java
      javac -source 8 -target 8 /tmp/pip-install-gv_sm6fr/python-javabridge_ea4288a3fc0b4e16815944ed67e58092/java/org/cellprofiler/javabridge/test/RealRect.java
      javac -source 8 -target 8 /tmp/pip-install-gv_sm6fr/python-javabridge_ea4288a3fc0b4e16815944ed67e58092/java/org/cellprofiler/javabridge/CPython.java /tmp/pip-install-gv_sm6fr/python-javabridge_ea4288a3fc0b4e16815944ed67e58092/java/org/cellprofiler/javabridge/CPythonInvocationHandler.java
      Note: /tmp/pip-install-gv_sm6fr/python-javabridge_ea4288a3fc0b4e16815944ed67e58092/java/org/cellprofiler/javabridge/CPythonInvocationHandler.java uses unchecked or unsafe operations.
      Note: Recompile with -Xlint:unchecked for details.
      building 'javabridge._javabridge' extension
      creating build/temp.linux-x86_64-cpython-311
      gcc -pthread -B /path/to/envs/javabridge-install/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /path/to/envs/javabridge-install/include -fPIC -O2 -isystem /path/to/envs/javabridge-install/include -fPIC -I/path/to/envs/javabridge-install/lib/python3.11/site-packages/numpy/core/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-2.el8.x86_64/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-2.el8.x86_64/include/linux -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-2.el8.x86_64/default-java/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-2.el8.x86_64/default-java/include/linux -I/path/to/envs/javabridge-install/lib/python3.11/site-packages/numpy/core/include -I/path/to/envs/javabridge-install/include/python3.11 -c _javabridge.c -o build/temp.linux-x86_64-cpython-311/_javabridge.o
      _javabridge.c:196:12: fatal error: longintrepr.h: No such file or directory
         #include "longintrepr.h"
                  ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for python-javabridge
  Running setup.py clean for python-javabridge
Failed to build python-javabridge
ERROR: Could not build wheels for python-javabridge, which is required to install pyproject.toml-based projects

现咨询：

1. Python3.10与3.11之间pip/setuptools的安装机制有何变化？
2. 如何修复该错误使python-javabridge兼容Python3.11及更高版本？

一、Python3.10与3.11的pip/setuptools安装机制变化 #

1. PEP 517/518成为默认构建标准
   Python3.11开始，pip默认采用PEP 517规范构建包，不再直接执行setup.py，而是通过pyproject.toml定义的构建后端处理流程。Python3.10虽已支持PEP 517，但仍允许传统setup.py直接构建，未强制切换。
2. setuptools旧接口废弃
   setuptools适配Python3.11时，移除了setuptools.installer模块，标记fetch_build_eggs为废弃接口，强制要求使用PEP 517兼容的依赖管理方式，依赖旧接口的项目会触发警告或构建失败。
3. C扩展构建的底层限制收紧
   Python3.11调整了内部C API，移除了longintrepr.h这类非公开内部头文件，PEP 517构建流程更严格遵循公开C API规范，禁止访问内部私有头文件，这是导致编译错误的直接关联因素。

二、修复python-javabridge兼容Python3.11及更高版本的步骤 #

1. 修复longintrepr.h缺失的编译错误 #

Python3.11将longintrepr.h的内容合并到pycore_long.h中，需修改_javabridge.c文件：

• 找到#include "longintrepr.h"行，替换为版本兼容的条件引入：

  #if PY_VERSION_HEX >= 0x030B0000
  #include "pycore_long.h"
  #else
  #include "longintrepr.h"
  #endif
  

• 检查代码中依赖longintrepr.h的结构体（如PyLongObject），确保符合Python3.11的C API规范，避免因内部结构调整导致的异常。

2. 适配PEP 517构建流程 #

在项目根目录添加pyproject.toml文件，定义构建依赖和后端：

[build-system]
requires = ["setuptools>=60.0", "wheel", "numpy>=1.20.1"]
build-backend = "setuptools.build_meta"

该文件会引导pip使用PEP 517兼容方式构建，避免触发旧接口的废弃警告。

3. 调整setup.py中的废弃代码 #

如果setup.py中使用了dist.fetch_build_eggs，移除该调用，将构建依赖通过pyproject.toml声明，或改用setuptools.setup的setup_requires参数。

4. 本地构建测试 #

修改完成后，在Python3.11环境执行以下命令验证：

# 进入修改后的项目目录
cd python-javabridge
# 创建并激活测试环境
mamba create -n javabridge-py311 python==3.11.8
conda activate javabridge-py311
# 安装构建依赖
pip install setuptools>=60.0 wheel numpy>=1.20.1
# 构建并安装
pip install .

5. 兼容性验证 #

安装完成后运行项目测试用例，排查其他Python3.11 C API的兼容性问题，比如：

• 已废弃的C函数调用
• 参数或返回值类型的变化
• 内存管理逻辑的调整

内容的提问来源于stack exchange，提问作者TheEponymousProgrammer