AKS中Azure DevOps Docker构建代理npm build失败求助

问题 #

我们在AKS中运行Azure DevOps构建代理，已通过Dockerfile为Ubuntu代理安装Node.js与npm，同时流水线的Install Node.js任务成功安装了node 16.14.2。但执行npm build时失败，报错显示node-gyp无法下载node-v16.14.2-headers.tar.gz（Pod无网络访问外部GitHub等地址）。我们有两个疑问：

1. 为何会重复触发npm安装？
2. 是否可以跳过npm安装步骤？

相关Dockerfile命令 #

#install node
RUN curl -sL https://deb.nodesource.com/setup_11.x  | bash -
RUN apt-get -y install nodejs
RUN npm install
RUN apt-get install g++ build-essential

Install Node.js任务输出 #

Found tool in cache: node 16.14.2 x64
Prepending PATH environment variable with directory: /azp/_work/_tool/node/16.14.2/x64/bin

错误日志 #

v16.14.2
8.5.0
npm cache location
/mnt/npmrepo
cleaning up workspace
Installing angular/cli

changed 214 packages in 7s
Running npm install
npm WARN deprecated source-map-resolve@0.6.0: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated messageformat@2.3.0: Package renamed as '@messageformat/core', see messageformat.github.io for more details. 'messageformat@4' will eventually provide a polyfill for Intl.MessageFormat, once it's been defined by Unicode & ECMA.
npm WARN deprecated protractor@7.0.0: We have news to share - Protractor is deprecated and will reach end-of-life by Summer 2023. To learn more and find out about other options please refer to this post on the Angular blog. Thank you for using and contributing to Protractor. https://goo.gle/state-of-e2e-in-angular
npm WARN deprecated core-js@3.20.3: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.
npm ERR! code 1
npm ERR! path /azp/_work/1/s/node_modules/fibers
npm ERR! command failed
npm ERR! command sh -c node build.js || nodejs build.js
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@8.4.1
npm ERR! gyp info using node@16.14.2 | linux | x64
npm ERR! gyp info find Python using Python version 3.6.9 found at "/usr/bin/python3"
npm ERR! gyp http GET https://nodejs.org/download/release/v16.14.2/node-v16.14.2-headers.tar.gz
npm ERR! gyp WARN install got an error, rolling back install
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack FetchError: request to https://nodejs.org/download/release/v16.14.2/node-v16.14.2-headers.tar.gz failed, reason: read ECONNRESET
npm ERR! gyp ERR! stack     at ClientRequest.<anonymous> (/azp/_work/1/s/node_modules/minipass-fetch/lib/index.js:110:14)
npm ERR! gyp ERR! stack     at ClientRequest.emit (node:events:526:28)
npm ERR! gyp ERR! stack     at TLSSocket.socketErrorListener (node:_http_client:442:9)
npm ERR! gyp ERR! stack     at TLSSocket.emit (node:events:538:35)
npm ERR! gyp ERR! stack     at emitErrorNT (node:internal/streams/destroy:157:8)
npm ERR! gyp ERR! stack     at emitErrorCloseNT (node:internal/streams/destroy:122:3)
npm ERR! gyp ERR! stack     at processTicksAndRejections (node:internal/process/task_queues:83:21)
npm ERR! gyp ERR! System Linux 5.4.0-1086-azure
npm ERR! gyp ERR! command "/azp/_work/_tool/node/16.14.2/x64/bin/node" "/azp/_work/1/s/node_modules/.bin/node-gyp" "rebuild" "--release"
npm ERR! gyp ERR! cwd /azp/_work/1/s/node_modules/fibers
npm ERR! gyp ERR! node -v v16.14.2
npm ERR! gyp ERR! node-gyp -v v8.4.1
npm ERR! gyp ERR! not ok 
npm ERR! node-gyp exited with code: 1
npm ERR! Please make sure you are using a supported platform and node version. If you
npm ERR! would like to compile fibers on this machine please make sure you have setup your
npm ERR! build environment--
npm ERR! Windows + OS X instructions here: https://github.com/nodejs/node-gyp
npm ERR! Ubuntu users please run: `sudo apt-get install g++ build-essential`
npm ERR! RHEL users please run: `yum install gcc-c++` and `yum groupinstall 'Development Tools'` 

npm ERR! gyp WARN install got an error, rolling back install
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack FetchError: request to https://nodejs.org/download/release/v16.14.2/node-v16.14.2-headers.tar.gz failed, reason: read ECONNRESET
npm ERR! gyp ERR! stack     at ClientRequest.<anonymous> (/azp/_work/1/s/node_modules/minipass-fetch/lib/index.js:110:14)
npm ERR! gyp ERR! stack     at ClientRequest.emit (node:events:526:28)
npm ERR! gyp ERR! stack     at TLSSocket.socketErrorListener (node:_http_client:442:9)
npm ERR! gyp ERR! stack     at TLSSocket.emit (node:events:538:35)
npm ERR! gyp ERR! stack     at emitErrorNT (node:internal/streams/destroy:157:8)
npm ERR! gyp ERR! stack     at emitErrorCloseNT (node:internal/streams/destroy:122:3)
npm ERR! gyp ERR! stack     at processTicksAndRejections (node:internal/process/task_queues:83:21)
npm ERR! gyp ERR! System Linux 5.4.0-1086-azure
npm ERR! gyp ERR! command "/azp/_work/_tool/node/16.14.2/x64/bin/node" "/azp/_work/1/s/node_modules/.bin/node-gyp" "rebuild" "--release"
npm ERR! gyp ERR! cwd /azp/_work/1/s/node_modules/fibers
npm ERR! gyp ERR! node -v v16.14.2
npm ERR! gyp ERR! node-gyp -v v8.4.1
npm ERR! gyp ERR! not ok 
npm ERR! node-gyp exited with code: 1
npm ERR! Please make sure you are using a supported platform and node version. If you
npm ERR! would like to compile fibers on this machine please make sure you have setup your
npm ERR! build environment--
npm ERR! Windows + OS X instructions here: https://github.com/nodejs/node-gyp
npm ERR! Ubuntu users please run: `sudo apt-get install g++ build-essential`
npm ERR! RHEL users please run: `yum install gcc-c++` and `yum groupinstall 'Development Tools'` 
npm ERR! Alpine users please run: `sudo apk add python make g++`

npm ERR! A complete log of this run can be found in:
npm ERR!     /mnt/npmrepo/_logs/2022-11-16T14_47_37_182Z-debug-0.log

解决方案 #

1. 重复触发npm install的原因 #

• Dockerfile中的RUN npm install是在镜像构建阶段执行的，针对的是镜像内的当前工作目录，但流水线运行时，项目代码是挂载到/azp/_work/1/s目录的，这个目录在镜像构建时不存在，所以镜像里安装的依赖和流水线工作目录的依赖完全无关。
• 流水线中的npm install是构建任务中明确配置的步骤（或是Angular CLI构建默认触发的依赖安装），和Dockerfile里的步骤是独立的两个操作。

2. 跳过npm安装步骤的方法 #

可以跳过，但需要提前准备好项目依赖：

• 方法一：在Dockerfile中预安装项目依赖
  将项目的package.json和package-lock.json复制到镜像中，再执行npm install，把依赖打包进镜像：

  # 先复制依赖锁文件，保证版本一致性
  COPY package*.json ./
  RUN npm install
  # 再复制项目代码
  COPY . .
  

  这样流水线中直接执行npm run build即可，无需再执行npm install。
• 方法二：在流水线中禁用npm install步骤
  如果使用Azure DevOps的Angular模板任务，找到任务配置选项，取消勾选"Install dependencies"类的选项；如果是自定义脚本，直接删除npm install命令，只保留构建命令。

3. 解决node-gyp下载失败的问题（网络受限场景） #

因为Pod无法访问外部网络，node-gyp无法下载Node.js headers，可通过以下方式解决：

• 提前在镜像中下载headers包
  在Dockerfile中手动下载对应版本的headers，然后设置环境变量让node-gyp使用本地文件：

  # 下载headers到镜像内指定路径
  RUN curl -O https://nodejs.org/download/release/v16.14.2/node-v16.14.2-headers.tar.gz -L
  # 设置node-gyp使用本地headers
  ENV npm_config_headers="/node-v16.14.2-headers.tar.gz"
  

• 使用内部镜像源
  配置npm使用私有内部镜像源，同时将node-gyp需要的headers包同步到内部源，让node-gyp从内部源下载文件，无需访问外部网络。

内容的提问来源于stack exchange，提问作者Vowneee