启用CSFLE的Node.js MongoDB企业版数据未加密问题

在本地开发环境中，使用Compass连接启用了CSFLE的MongoDB Enterprise客户端时，所有本该通过JSON schema加密的字段都以明文显示。

背景信息 #

• 开发环境使用MongoDB Enterprise，生产部署采用Atlas Cloud；
• 计划生产环境用AWS KMS作为密钥，本地已尝试用本地密钥和AWS IAM用户凭证实现自动客户端字段级加密（CSFLE）；
• 已确认mongocryptd进程正在运行；
• 服务器启动时连接多个对应不同微服务的MongoDB实例，加密凭证在该服务器配置；
• 微服务接收DEK并根据自身需求创建schema map；
• 通过mongosh查看MongoDB日志未发现错误；
• 已删除数据库并重新初始化；
• 使用mongoose框架。

MongoDB版本 #

MongoDB Enterprise 7.0.2

Node 18依赖包版本 #

"mongodb": "^6.2.0",
"mongodb-client-encryption": "^6.0.0",
"mongoose": "^8.0.0",

CSFLE初始化流程 #

1. 在服务器顶层调用Encryption类的initialize方法，该方法返回包含DEK等通用配置参数的对象供微服务使用，创建加密客户端、密钥库集合（不存在则创建）并生成DEK，之后关闭连接；
2. 将返回的配置参数传入微服务初始化方法，创建无服务端加密schema的数据库模型，并通过autoEncryption配置参数为每个服务创建常规MongoDB客户端连接。

代码实现 #

注：已移除冗余代码以简化展示 #

CSFLE辅助类 #

export default class Encryption implements IEncryption {
  // ... 此处省略若干私有和公共变量

  // 私有构造函数，需通过下方initialize方法调用
  private constructor(opts?: EncryptionConfigConstructorOpts) {
    this.tenantId = opts?.tenantId;
    this.keyVaultDbName = opts?.keyVaultDbName;
    this.keyVaultCollectionName = opts?.keyVaultCollectionName;
    this.DEKAlias = opts?.DEKAlias;

    // 检测本地开发环境
    if (process.env?.ENVIRONMENT === LOCAL_DEV_ENV) {
      const keyBase64 = process.env?.LOCAL_MASTER_KEY;
      const key = Buffer.from(keyBase64, 'base64');

      // 测试时手动切换本地密钥与远程KMS，此处省略生产环境检测代码
      if (_debug) {
        this.provider = KMS_PROVIDER;
        this.kmsProviders = {
          aws: {
            accessKeyId: process.env.AWS_ACCESS_KEY_ID,
            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
          },
        };
        this.masterKey = {
          key: process.env.KMS_MASTER_ARN,
          region: opts?.masterRegion,
        };
      } else {
        this.kmsProviders = {
           local: {
             key,
           },
        };
      }
    }

    const keyVaultNamespace = `${this.keyVaultDbName}.${this.keyVaultCollectionName}`;

    const encryptionOptions: ClientEncryptionOptions = {
      keyVaultNamespace,
      kmsProviders: this.kmsProviders,
    };

    this.encryptionOptions = encryptionOptions;
  }

  public static async initialize(
    url: string,
    opts?: EncryptionConfigConstructorOpts
  ): Promise<Encryption> {
    // 设置内部属性
    const encryption = new Encryption(opts);

    // 创建密钥库集合（幂等操作）
    const client = new MongoClient(url);
    const keyVaultDB = client.db(encryption.keyVaultDbName);
    const keyVaultColl = keyVaultDB.collection(encryption.keyVaultCollectionName);
    await keyVaultColl.createIndex(
      { keyAltNames: 1 },
      {
        unique: true,
        partialFilterExpression: { keyAltNames: { $exists: true } },
      }
    );

    let dek: UUID | undefined = undefined;

    // 检查现有DEK，不存在则创建，存在则直接使用
    try {
      // 初始化客户端加密
      const clientEncryption = new ClientEncryption(client, encryption.encryptionOptions!);
      const keyOptions = {
        masterKey: encryption.masterKey,
        keyAltNames: [encryption.DEKAlias],
      };
      dek = await clientEncryption.createDataKey(encryption.provider, keyOptions);
    } catch (err: any) {
      // 重复键错误为预期情况，此时获取已有密钥
      if (String(err?.code) !== '11000') {
        throw err;
      } else {
        // 检查是否存在对应DEK别名的密钥
        const existingKey = await client
          .db(encryption.keyVaultDbName)
          .collection(encryption.keyVaultCollectionName)
          .findOne({ keyAltNames: encryption.DEKAlias });

        if (existingKey?._id) {
          dek = UUID.createFromHexString(existingKey._id.toHexString());
        } else {
          throw new Error('DEK could not be found or created');
        }
      }
    } finally {
      await client.close();
    }

    encryption.dek = dek;
    encryption.isReady = !!encryption.dek;
    return encryption;
  }

  // 箭头函数以保留this上下文，供微服务回调调用
  // 在initialize方法后由各微服务调用
  public getSchemaMap = (
    jsonSchema: Record<string, unknown>,
    encryptionMetadata?: Record<string, unknown>
  ): Record<string, unknown> => {
    if (!this?.isReady) {
      throw new Error('Encryption class cannot get schema map until it is initialized');
    }

    const schemaMapWithEncryption = {
      encryptMetadata: {
        keyId: [this.dek],
        algorithm: process.env.ALG_DETERMINISTIC,
        ...encryptionMetadata,
      },
      ...jsonSchema,
    };
    return schemaMapWithEncryption;
  };
}

启动代码 #

// ... 启动代码省略
    const encryption = await Encryption.initialize(process.env.DB_CONN_STRING);
    const opts = {
      autoEncryption: {
        ...encryption.encryptionOptions
      },
    };

    await Service1Models.initialize(process.env.DB_CONN_STRING, opts, encryption.getSchemaMap);
    await Service2Models.initialize(process.env.DB_CONN_STRING, opts, encryption.getSchemaMap);

// ... 更多启动代码及API路由配置省略

典型微服务initialize方法 #

// ... 初始化代码省略，之后生成无加密语法的模型
Service1Model.service1DataModel = model<IService1Document>('Service1Doc', Service1Schema, 'Service1Docs');

// 最终使用为该服务生成的schema map连接数据库
mongoose.connect(url, {
        ...opts,
        autoEncryption: opts?.autoEncryption
          ? {
              ...opts?.autoEncryption,
              schemaMap: getSchemaMap(importedSchemaJson),
            }
          : undefined,
      } as ConnectOptions);

加密JSON Schema #

{
  "MyCollection1": {
    "properties": {
      "myDataString": {
        "encrypt": {
          "bsonType": "string"
        }
      },
      "myDataArray": {
        "encrypt": {
          "bsonType": "array"
        }
      },
      "myDataObject": {
        "bsonType": "object",
        "properties": {
          "myNestedProperty1": {
            "encrypt": {
              "bsonType": "string"
            }
          },
          "myNestedProperty2": {
            "bsonType": "string"
          }
        }
      }
    }
  }
}

内容的提问来源于stack exchange，提问作者Angus Ryer