GitLab Pipeline无代码变更突发失败 node-gyp报Python缺失错误

推送代码到企业组织自建GitLab实例后流水线执行失败，经多轮验证确认故障与本次代码变更无关：重跑过往可成功执行的旧提交对应流水线，出现完全相同的报错，排除本次代码提交的影响。
GitLab Runner部署在Kubernetes集群上，流水线基于node:16-alpine Docker镜像运行。

.gitlab-ci.yml 配置 #

stages:
  - cli_build_and_analyze
  - some other stages

job_build_and_analyze:
  stage: cli_build_and_analyze
  image: artifacts.company.com/docker-mirror/node:16-alpine
  tags:
    - linux-k8s
  script:
    - export http_proxy="http://proxy.company.com:8080"
    - export https_proxy="http://proxy.company.com:8080"
    - export no_proxy=".company.com"
    - node --version
    - cd cli
    - npm config set registry https://artifacts.company.com/artifactory/api/npm/npm-mirror/
    - npm install
    - npx pkg-fetch -n node16 -p win -a x64
    - yarn build
    - yarn coverage
  artifacts:
    paths:
      - cli/*
    expire_in: 4 h

some other stages: ...

项目依赖配置 #

{
  "devDependencies": {
    "@types/jest": "^27.4.0",
    "@types/node": "^17.0.8",
    "jest": "^27.4.7",
    "ts-jest": "^27.1.2",
    "typescript": "^4.4.4"
  },
  "dependencies": {
    "commander": "^8.3.0",
    "esbuild": "^0.14.11",
    "node-opcua": "^2.62.7",
    "pkg": "^5.5.1"
  }
}

Running with gitlab-runner 14.5.0 (f0a95a76)
  on server-k8s 9-Y7h2DK
Resolving secrets
00:00
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: opcerver
Using Kubernetes executor with image artifacts.company.com/docker-mirror/node:16-alpine ...
Using attach strategy to execute scripts...
Preparing environment
00:03
Waiting for pod server/runner-9-y7h2dk-project-1654-concurrent-0bck9f to be running, status is Pending
    ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
    ContainersNotReady: "containers with unready status: [build helper]"
    ContainersNotReady: "containers with unready status: [build helper]"
Running on runner-9-y7h2dk-project-1653-concurrent-0bck9f via gitlab-runner-gitlab-runner-6965989c6b-b6l9x...
Getting source from Git repository
00:01
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/teams/teamname/validation-tool/.git/
Created fresh repository.
Checking out 920d7008 as accept-regex...
Skipping Git submodules setup
Executing "step_script" stage of the job script
01:13
$ export http_proxy="http://proxy.company.com:8080"
$ export https_proxy="http://proxy.company.com:8080"
$ export no_proxy=".company.com"
$ node --version
v16.15.0
$ cd cli
$ npm config set registry https://artifacts.company.com/artifactory/api/npm/npm-mirror/
$ npm install
npm ERR! code 1
npm ERR! path /builds/teams/team/validation-tool/cli/node_modules/bufferutil
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.0.0
npm ERR! gyp info using node@16.15.0 | linux | x64
npm ERR! gyp ERR! find Python 
npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration
npm ERR! gyp ERR! find Python Python is not set from environment variable PYTHON
npm ERR! gyp ERR! find Python checking if "python3" can be used
npm ERR! gyp ERR! find Python - "python3" is not in PATH or produced an error
npm ERR! gyp ERR! find Python checking if "python" can be used
npm ERR! gyp ERR! find Python - "python" is not in PATH or produced an error
npm ERR! gyp ERR! find Python 
npm ERR! gyp ERR! find Python **********************************************************
npm ERR! gyp ERR! find Python You need to install the latest version of Python.
npm ERR! gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
npm ERR! gyp ERR! find Python you can try one of the following options:
npm ERR! gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
npm ERR! gyp ERR! find Python   (accepted by both node-gyp and npm)
npm ERR! gyp ERR! find Python - Set the environment variable PYTHON
npm ERR! gyp ERR! find Python - Set the npm configuration variable python:
npm ERR! gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
npm ERR! gyp ERR! find Python For more information consult node-gyp official installation documentation.
npm ERR! gyp ERR! find Python **********************************************************
npm ERR! gyp ERR! find Python 
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: Could not find any Python installation to use
npm ERR! gyp ERR! stack     at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:330:47)
npm ERR! gyp ERR! stack     at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:159:21)
npm ERR! gyp ERR! stack     at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:202:16)
npm ERR! gyp ERR! stack     at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:294:16)
npm ERR! gyp ERR! stack     at exithandler (node:child_process:406:5)
npm ERR! gyp ERR! stack     at ChildProcess.errorhandler (node:child_process:418:5)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:527:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:289:12)
npm ERR! gyp ERR! stack     at onErrorNT (node:internal/child_process:478:16)
npm ERR! gyp ERR! stack     at processTicksAndRejections (node:internal/process/task_queues:83:21)
npm ERR! gyp ERR! System Linux 4.12.14-122.106-default
npm ERR! gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /builds/teams/team/validation-tool/cli/node_modules/bufferutil
npm ERR! gyp ERR! node -v v16.15.0
npm ERR! gyp ERR! node-gyp -v v9.0.0
npm ERR! gyp ERR! not ok
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-07-06T11_27_26_562Z-debug-0.log
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1

• 移除所有依赖版本号前的^符号，锁定依赖精确版本，仍复现报错
• 测试更高版本Node镜像、其他类型基础镜像，均复现相同报错
• 确认近三周未修改任何流水线相关配置，该流水线三周前可正常运行，连续排查四天未定位根因

核心原因是CI使用的基础镜像标签node:16-alpine是浮动标签，私有镜像仓库最近同步了上游更新后的镜像版本，新版本Alpine基础镜像默认不再预装Python3、make、C/C++编译工具链等原生模块编译依赖。
npm install执行时需要编译bufferutil这类原生Node模块，依赖node-gyp，而node-gyp运行必须有Python环境和编译工具链，镜像里缺少这些依赖就会直接报找不到Python的错误。这也是旧提交重跑、换更高版本Node镜像仍报错的原因——所有未固定版本的alpine标签Node镜像更新后都默认不带这些编译依赖。

• 临时快速修复：在npm install命令前添加依赖安装命令，补全编译所需系统组件：

apk add --no-cache python3 make g++
ln -sf /usr/bin/python3 /usr/bin/python

• 长期稳定修复：
  • 不要使用浮动版本的镜像标签，将CI基础镜像固定到明确的版本号，例如artifacts.company.com/docker-mirror/node:16.14.2-alpine3.15，避免上游镜像非预期更新导致流水线故障
  • 自行构建预安装了编译依赖、代理配置、npm源配置的专用CI基础镜像推送到私有仓库，无需每次流水线执行时重复安装系统依赖，提升执行稳定性和速度

注：bufferutil是node-opcua的可选性能依赖，若不需要极致性能，也可在npm install时添加--no-optional参数跳过可选依赖安装，绕过原生模块编译步骤，但优先推荐前两种修复方案。

内容的提问来源于stack exchange，提问作者Robin Aegerter