如何解决Nix构建Python包时DocArray与Jina的循环依赖问题

问题本质 #

Jina安装依赖DocArray，而DocArray的部分子依赖又依赖Jina生态组件，形成循环依赖链，导致Nix的依赖解析器无法完成构建。

解决方案 #

1. 指定兼容版本并手动调整依赖 #

锁定DocArray和Jina的兼容版本，通过overrideAttrs修改依赖列表，绕过循环：

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
        python = pkgs.python311;
        pythonEnv = python.withPackages (ps: with ps; [
          # 调整DocArray依赖，跳过不必要的Jina关联检查
          (docarray.overrideAttrs(old: {
            propagatedBuildInputs = builtins.filter (p: !builtins.match "jina.*" p.name) old.propagatedBuildInputs;
            propagatedBuildInputs = old.propagatedBuildInputs ++ [ pydantic requests ];
            doCheck = false; # 可选：跳过测试加速构建
          }))
          # 剥离Jina对DocArray的自动依赖，手动添加指定版本
          (jina.overrideAttrs(old: {
            propagatedBuildInputs = builtins.filter (p: p.name != "docarray") old.propagatedBuildInputs;
            propagatedBuildInputs = old.propagatedBuildInputs ++ [ docarray ];
          }))
          jina-hubble-sdk
        ]);
      in {
        devShells.default = pkgs.mkShell {
          buildInputs = [ pythonEnv ];
        };
      }
    );
}

2. 剥离循环依赖项 #

针对循环的核心组件，手动移除其中一方的依赖，再手动补全环境所需包：

# 示例：剥离Jina对DocArray的依赖，之后单独添加DocArray
ps.jina.overridePythonAttrs(oldAttrs: {
  propagatedBuildInputs = builtins.filter (p: p.name != "docarray") oldAttrs.propagatedBuildInputs;
})

在withPackages列表中同时加入处理后的Jina和原生DocArray，确保两者都存在于环境中。

3. 临时方案：用pip直接安装 #

如果Nix原生解析难以调试，可使用buildPythonPackage调用pip安装，绕过Nix依赖解析：

(pkgs.python311Packages.buildPythonPackage rec {
  pname = "jina-docarray-env";
  version = "0.1";
  src = null;
  buildInputs = [ pkgs.python311Packages.pip ];
  propagatedBuildInputs = [ pkgs.python311Packages.setuptools ];
  doCheck = false;
  installPhase = ''
    pip install jina docarray jina-hubble-sdk --no-cache-dir
  '';
})

注意：此方法会丢失Nix的依赖隔离优势，仅适合临时测试场景。

验证 #

构建环境后，进入shell执行以下命令确认安装成功：

python -c "import docarray; import jina; from jina_hubble_sdk import Client; print('所有包加载成功')"

内容的提问来源于stack exchange，提问作者Pierre-olivier Gendraud