使用环境标记安装平台依赖包失败问题排查

背景信息 #

• 有两个自研Python包foo和bar
• foo是平台依赖包，仅支持Windows平台，依赖pywin32
• bar是跨平台包，仅在Windows环境下需要依赖foo
• 尝试通过环境标记配置bar的依赖，让foo仅在Windows平台安装，但安装时出现元数据生成错误

foo的pyproject.toml配置 #

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "foo"
version = "0.0.1"
authors = [
    { name="John Doe", email="John.Doe@example.com" },
]
description = "FOO"
requires-python = ">=3.7"
dependencies = [
    "pywin32"
]
classifiers = [
    "Programming Language :: Python :: 3",
    "Operating System :: Microsoft :: Windows"
]

bar的原pyproject.toml配置 #

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "bar"
version = "0.0.1"
authors = [
    { name="John Doe", email="John.Doe@example.com" },
]
description = "BAR"
requires-python = ">=3.7"
dependencies = [
    "foo @ file:///C:/Users/bkm/Workspace/temp/foo; platform_system == 'Windows'"
]
classifiers = [
    "Programming Language :: Python :: 3",
    "Operating System :: OS Independent"
]

[tool.hatch.metadata]
allow-direct-references = true

安装时的错误日志 #

Obtaining file:///C:/Users/bkm/Workspace/temp/bar
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing editable metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      Traceback (most recent call last):
        File "C:\Users\bkm\Workspace\temp\bar\winenv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\bkm\Workspace\temp\bar\winenv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bkm\Workspace\temp\bar\winenv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 176, in prepare_metadata_for_build_editable
          whl_basename = build_hook(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bkm\AppData\Local\Temp\pip-build-env-rz8bsp6f\overlay\Lib\site-packages\hatchling\build.py", line 83, in build_editable
          return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bkm\AppData\Local\Temp\pip-build-env-rz8bsp6f\overlay\Lib\site-packages\hatchling\builders\plugin\interface.py", line 90, in build
          self.metadata.validate_fields()
        File "C:\Users\bkm\AppData\Local\Temp\pip-build-env-rz8bsp6f\overlay\Lib\site-packages\hatchling\metadata\core.py", line 266, in validate_fields
          self.core.validate_fields()
        File "C:\Users\bkm\AppData\Local\Temp\pip-build-env-rz8bsp6f\overlay\Lib\site-packages\hatchling\metadata\core.py", line 1376, in validate_fields
          getattr(self, attribute)
        File "C:\Users\bkm\AppData\Local\Temp\pip-build-env-rz8bsp6f\overlay\Lib\site-packages\hatchling\metadata\core.py", line 1230, in dependencies
          self._dependencies = list(self.dependencies_complex)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\bkm\AppData\Local\Temp\pip-build-env-rz8bsp6f\overlay\Lib\site-packages\hatchling\metadata\core.py", line 1208, in dependencies_complex
          raise ValueError(message) from None
      ValueError: Dependency #1 of field `project.dependencies` is invalid: Expected end or semicolon (after URL and whitespace)
          foo @ file:///C:/Users/bkm/Workspace/temp/foo; platform_system == 'Windows'
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.1.2 -> 24.2
[notice] To update, run: python.exe -m pip install --upgrade pip

解决方案 #

1. 修正依赖格式错误 #

错误根源是PEP 508格式要求：当本地依赖（URL形式）搭配环境标记时，URL和分号之间必须添加空格。

修改bar的pyproject.toml中dependencies字段：

dependencies = [
    "foo @ file:///C:/Users/bkm/Workspace/temp/foo ; platform_system == 'Windows'"
]

2. 可选方案：使用可选依赖配合环境标记 #

如果需要更清晰的依赖管理逻辑，可以将foo设为bar的可选依赖，再通过Hatchling的环境标记自动关联平台：

[project]
# 保留其他原有配置
dependencies = []

[project.optional-dependencies]
windows = ["foo @ file:///C:/Users/bkm/Workspace/temp/foo"]

[tool.hatch.metadata.environment-markers]
extra == "windows" = "platform_system == 'Windows'"

这种配置下，pip会在Windows平台自动识别并安装windows可选依赖组。

内容的提问来源于stack exchange，提问作者Bijesh Mohan