使用Poetry与CDK Synth时，私有PyPI仓库凭证传递问题

问题描述 #

在GitHub Action中通过AWS CDK部署依赖私有PyPI仓库的Python Lambda Layer时，cdk synth步骤失败，报错：

ERROR: Could not find a version that satisfies the requirement mypackage0.1.1 (from versions: none)
ERROR: No matching distribution found for mypackage0.1.1

此前的Action步骤（安装npm/CDK、Python/Poetry及依赖、测试、cdk bootstrap）均正常，且非CDK的Poetry步骤中，通过POETRY_HTTP_BASIC_*环境变量配置私有仓库凭证是有效的。Lambda Layer目录下的pyproject.toml与poetry.lock已从之前的Poetry步骤复制，配置如下：

[tool.poetry.dependencies]
mypackage = {version="*", source = "internal"}
[[tool.poetry.source]]
name = "internal"
url = "..."
priority = "explicit" 

核心原因 #

CDK在构建Python Lambda Layer时，会启动独立的Docker容器执行依赖安装流程，GitHub Action步骤中直接设置的环境变量不会自动传递到Docker容器内部，导致Poetry无法获取私有仓库的访问凭证，进而拉取包失败。

解决方案 #

方法一：通过CDK代码传递环境变量到Docker构建 #

在CDK的PythonLayerVersion构造中，使用build_args参数将凭证环境变量传递给Docker构建环境：

import os
from aws_cdk import aws_lambda as lambda_
from constructs import Construct

class MyStack(Construct):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)
        
        # 构建Lambda Layer时传递私有仓库凭证
        lambda_layer = lambda_.PythonLayerVersion(
            self, "MyPrivateLayer",
            entry="path/to/your/layer/directory",
            build_args={
                "POETRY_HTTP_BASIC_INTERNAL_USERNAME": os.environ["POETRY_HTTP_BASIC_INTERNAL_USERNAME"],
                "POETRY_HTTP_BASIC_INTERNAL_PASSWORD": os.environ["POETRY_HTTP_BASIC_INTERNAL_PASSWORD"]
            }
        )

同时保留GitHub Action中cdk synth步骤的环境变量配置，确保CDK代码能读取到这些变量：

- name: CDK Synth
  env:
    POETRY_HTTP_BASIC_INTERNAL_PASSWORD: ${{ secrets.relevant_secret }}
    POETRY_HTTP_BASIC_INTERNAL_USERNAME: ${{ secrets.other_relevant_secret }}
  run: |
    npx aws-cdk synth --validation --verbose

方法二：在Layer目录生成带凭证的Poetry配置文件 #

在GitHub Action的Poetry依赖安装步骤之后，生成包含私有仓库凭证的poetry.toml文件，复制到Lambda Layer目录中，让Docker内的Poetry自动读取：

- name: Generate Poetry auth config for Layer
  run: |
    # 生成包含私有仓库凭证的poetry配置文件
    cat > ./path/to/your/layer/directory/poetry.toml << EOF
    [http-basic.internal]
    username = "${{ secrets.other_relevant_secret }}"
    password = "${{ secrets.relevant_secret }}"
    EOF

注意：务必将poetry.toml添加到项目的.gitignore中，避免敏感凭证提交到代码仓库。

方法三：使用Docker BuildKit传递构建秘钥（更安全） #

启用Docker BuildKit后，可以通过构建秘钥传递凭证，避免明文暴露：

1. 在GitHub Action中启用BuildKit：

- name: CDK Synth
  env:
    DOCKER_BUILDKIT: 1
    POETRY_USERNAME: ${{ secrets.other_relevant_secret }}
    POETRY_PASSWORD: ${{ secrets.relevant_secret }}
  run: |
    npx aws-cdk synth --validation --verbose

2. 在CDK代码中配置build_secrets参数：

import os
from aws_cdk import aws_lambda as lambda_
from constructs import Construct

class MyStack(Construct):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)
        
        lambda_layer = lambda_.PythonLayerVersion(
            self, "MyPrivateLayer",
            entry="path/to/your/layer/directory",
            build_args={"DOCKER_BUILDKIT": "1"},
            build_secrets={
                "POETRY_USERNAME": f"env:POETRY_USERNAME",
                "POETRY_PASSWORD": f"env:POETRY_PASSWORD"
            }
        )

3. 在Layer目录的pyproject.toml中，通过环境变量引用秘钥：

[[tool.poetry.source]]
name = "internal"
url = "..."
priority = "explicit"
[http-basic.internal]
username = "${POETRY_USERNAME}"
password = "${POETRY_PASSWORD}"

内容的提问来源于stack exchange，提问作者Guillaume