如何在Pino传输转换流中复用同一个crypto.Cipheriv实例？

我正在使用Pino日志库，尝试对日志流进行加密并写入文件。目前通过创建传输管道转换数据并加密内容的方式可以正常工作，代码如下：

import build from "pino-abstract-transport";
import { pipeline, Transform } from "node:stream";
import crypto from "node:crypto";

const ALGORITHM = "aes-256-ctr";

export default async function (options: { password: string }) {
  const password = Buffer.from(options.password, "hex");

  const iv = crypto.randomBytes(16);

  return build(function (source) {
    const myTransportStream = new Transform({
      autoDestroy: true,
      objectMode: true,
      transform(chunk, _enc, end) {
        const encrypt = crypto.createCipheriv(ALGORITHM, password, iv);
        const data = encrypt.update(JSON.stringify(chunk));
        const encrypted = Buffer.concat([data, encrypt.final()]);

        this.push(encrypted.toString("hex") + '\n');

        end();
      },
    });
    pipeline(source,  myTransportStream, () => {});
    return myTransportStream;
  }, {
    enablePipelining: true,
  });
}

我想复用同一个const encrypt = crypto.createCipheriv(ALGORITHM, password, iv);实例，避免每次都创建新实例，同时想了解这样重构是否能提升性能？

我尝试了以下代码：

import build from "pino-abstract-transport";
import { pipeline, Transform } from "node:stream";
import crypto from "node:crypto";

const ALGORITHM = "aes-256-ctr";

export default async function (options: { password: string }) {
  let initiated = false;
  const password = Buffer.from(options.password, "hex");

  const iv = crypto.randomBytes(16);

  const encrypt = crypto.createCipheriv(ALGORITHM, password, iv);
  return build(function (source) {
    const myTransportStream = new Transform({
      autoDestroy: true,
      objectMode: true,
      transform(chunk, _enc, end) {
        if (!initiated) {
          initiated = true;
          this.push(Buffer.concat([iv, chunk]));
        } else {
          this.push(chunk);
        }

        end();
      },
    });
    pipeline(source, encrypt, myTransportStream, () => {});
    return myTransportStream;
  }, {
    enablePipelining: true,
  });
}

但出现了如下错误：

TypeError [ERR_INVALID_ARG_TYPE]: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of Object
    at new NodeError (node:internal/errors:372:5)
    at _write (node:internal/streams/writable:312:13)
    at Cipheriv.Writable.write (node:internal/streams/writable:334:10)
    at Transform.ondata (node:internal/streams/readable:754:22)
    at Transform.emit (node:events:527:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Transform.Readable.push (node:internal/streams/readable:228:10)
    at push (/mnt/spare/ent/back/Plugin-Stix-Core-API/node_modules/split2/index.js:76:10)
    at Transform.transform [as _transform] (/mnt/spare/ent/back/Plugin-Stix-Core-API/node_modules/split2/index.js:44:7)
Emitted 'error' event on ThreadStream instance at:

我不介意微优化，但如果能给出相关分析也欢迎。

我使用的是Fastify的Pino配置，与原生Pino配置基本一致：

transport: {
      pipeline: [
        {
          target: "./transform-log.js",
          options: {
            password:
              "f8647d5417039b42c88a75897109049378cdfce528a7e015656bd23cd18fb78a",
          },
        },
        {
          target: "pino/file",
          options: {
            destination: file,
          },
        },
      ],
    },

错误原因 #

Pino传输管道的上游source流输出的是日志对象，而crypto.Cipheriv属于字节流，只能处理字符串、Buffer或Uint8Array类型的数据，无法直接接收JavaScript对象，这就是报错的核心原因。

修复后的代码 #

我们需要先把日志对象序列化为JSON字符串，再传递给加密流，同时复用同一个Cipheriv实例，并且正确写入IV（解密时必须用到）：

import build from "pino-abstract-transport";
import { pipeline, Transform } from "node:stream";
import crypto from "node:crypto";

const ALGORITHM = "aes-256-ctr";

export default async function (options: { password: string }) {
  const password = Buffer.from(options.password, "hex");
  const iv = crypto.randomBytes(16);
  const encrypt = crypto.createCipheriv(ALGORITHM, password, iv);

  return build(function (source) {
    // 第一步：将日志对象转为JSON字符串并添加换行符
    const objectToStringStream = new Transform({
      autoDestroy: true,
      objectMode: true,
      transform(chunk, _enc, end) {
        const jsonStr = JSON.stringify(chunk) + '\n';
        this.push(jsonStr);
        end();
      }
    });

    // 第二步：处理IV的写入（只在开头写一次）
    const prependIvStream = new Transform({
      autoDestroy: true,
      transform(chunk, _enc, end) {
        // 第一次写入时先推送IV（转成十六进制方便存储）
        if (!this.hasWrittenIv) {
          this.hasWrittenIv = true;
          this.push(iv.toString('hex') + '\n');
        }
        this.push(chunk);
        end();
      }
    });

    // 构建完整管道：对象流 -> 字符串流 -> 加密流 -> 前置IV流
    pipeline(source, objectToStringStream, encrypt, prependIvStream, () => {});
    return prependIvStream;
  }, {
    enablePipelining: true,
  });
}

性能分析 #

复用Cipheriv实例确实能带来性能提升，原因如下：

1. 减少初始化开销：crypto.createCipheriv会执行加密算法的初始化步骤（比如AES的密钥扩展），这些操作是CPU密集型的，复用实例只需要执行一次初始化，而不是每条日志都重复执行。
2. 减少内存分配：每次创建新的Cipheriv实例都会分配新的内存和内部状态，复用实例能降低内存碎片和GC压力。

不过需要注意：

• CTR模式是流式加密，天然支持复用实例，不会出现数据混淆问题；但如果是其他模式（比如CBC），需要注意块对齐和填充问题，不适合直接流式复用。
• 整个日志文件使用同一个IV，符合CTR模式的安全要求（只要IV不重复使用同一个密钥即可），所以我们只需要在文件开头写入一次IV即可。

补充：解密示例 #

后续解密时，需要先读取文件开头的IV（十六进制字符串转Buffer），再用同一个密钥和IV创建Decipheriv实例，然后流式读取加密内容解密：

import { createReadStream, createWriteStream } from "node:fs";
import { pipeline, Transform } from "node:stream";
import crypto from "node:crypto";

const ALGORITHM = "aes-256-ctr";
const password = Buffer.from("你的十六进制密钥", "hex");

// 读取IV的流
const readIvStream = new Transform({
  transform(chunk, _enc, end) {
    if (!this.iv) {
      // 读取第一行作为IV
      const ivStr = chunk.toString().split('\n')[0];
      this.iv = Buffer.from(ivStr, 'hex');
      // 剩下的内容继续传递
      const remaining = chunk.slice(ivStr.length + 1);
      if (remaining.length > 0) this.push(remaining);
    } else {
      this.push(chunk);
    }
    end();
  }
});

// 延迟创建解密实例，确保获取到IV
readIvStream.once('data', () => {
  const decipher = crypto.createDecipheriv(ALGORITHM, password, readIvStream.iv);
  
  pipeline(
    createReadStream("加密的日志文件路径"),
    readIvStream,
    decipher,
    createWriteStream("解密后的日志文件路径"),
    (err) => {
      if (err) console.error("解密失败:", err);
      else console.log("解密完成");
    }
  );
});

内容的提问来源于stack exchange，提问作者Rick Stanley