NodeJS读取Google Cloud Bucket文件报错致Express重启问题排查

问题现象 #

在使用@google-cloud/storage的Express服务器中，部分场景下从Google Cloud Bucket通过可读流下载文件时，即便已监听流的.error事件（未触发）、且调用逻辑被try-catch包裹，仍会抛出未捕获错误导致Express实例重启（Pod未销毁）。

报错信息 #

报错栈如下：

TypeError: Cannot read properties of null (reading 'length')
    at getStateLength (/usr/src/node_modules/stream-shift/index.js:16:28)
    at shift (/usr/src/node_modules/stream-shift/index.js:6:99)
    at Duplexify._forward (/usr/src/node_modules/duplexify/index.js:170:35)
    at PassThrough.onreadable (/usr/src/node_modules/duplexify/index.js:136:10)
    at PassThrough.emit (node:events:518:28)
    at emitReadable_ (node:internal/streams/readable:832:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)

具体报错行是return state.buffer[0].length，对应stream-shift模块的第16行代码。

下载实现代码 #

const { bucketName, keyFilename } = config.google.storage;
if (!bucketName) {
  throw badImplementation('config.google.storage.bucketName is undefined');
}
if (!keyFilename) {
  throw badImplementation('config.google.storage.keyFilename is undefined');
}

const storage = new Storage({
  keyFilename,
  retryOptions: { autoRetry: true, maxRetries: 1 },
});
const bucket = storage.bucket(bucketName);

const [exists] = await bucket.file(name).exists();
if (!exists) {
  const error = `CDN download, file ${name} does not exist`;
  console.log(error);
  throw notFound(error);
}

log.info(`CDN download, create read stream on ${name} begin`);
const readStream = bucket
  .file(name)
  .createReadStream()
  .on('response', (response) => {
    // Server connected and responded with the specified status and headers.
    console.log(`CDN download, stream on file ${name}, response is: ${JSON.stringify(response)}`);
  })
  .on('end', () => {
    // The file is fully downloaded.
    console.log(`CDN download, stream on file ${name}, file fully downloaded`);
  })
  .on('error', (err) => {
    // Something happened while downloading the file
    console.log(`CDN download, stream on file ${name}, error is: ${JSON.stringify(err)}`);
  });

log.info(`CDN download, create read stream on ${name} done`);
return readStream;

排查过程 #

• 验证文件存在性：通过.exists()确认文件存在，且收到Google Cloud Storage返回的有效响应头：

{
  "headers": {
    "cache-control": "no-cache, no-store, max-age=0, must-revalidate",
    "content-disposition": "attachment",
    "content-length": "1309467",
    "content-type": "application/octet-stream",
    "date": "Wed, 24 Jan 2024 11:17:05 GMT",
    "etag": "CLSOmpWa8oMDEAE=",
    "expires": "Mon, 01 Jan 1990 00:00:00 GMT",
    "last-modified": "Tue, 23 Jan 2024 00:00:33 GMT",
    "pragma": "no-cache",
    "server": "UploadServer",
    "vary": "Origin, X-Origin",
    "x-goog-generation": "1705968033761076",
    "x-goog-hash": "crc32c=EeUAng==,md5=Duc9MjxstOaEXhEeZRphIw==",
    "x-goog-metageneration": "1",
    "x-goog-storage-class": "STANDARD",
    "x-goog-stored-content-encoding": "identity",
    "x-goog-stored-content-length": "1309467",
    "x-guploader-uploadid": "ABPtcPpJ0EZifzef-2dHFzbfURL0E_niJIylxjegZyJhjJ0kyhM8FGb7jymom35PJ4UrOcti3mp8CxNuqw"
  }
}

• 定位版本兼容问题：
  • 回退Docker镜像后发现，问题源于node:20镜像在1月9-11日的版本更新（从20.10升级到20.11）
  • 将Docker镜像切换为node:20.10.0后，问题完全解决，推测node:20.11.0存在相关兼容故障

官方排查请求 #

恳请Node.js或Google Cloud官方排查此问题，定位node:20.11.0与@google-cloud/storage流处理逻辑的冲突原因。

内容的提问来源于stack exchange，提问作者AlFranco