解决AWS Lambda并发执行时Mongoose Entity模型重复编译错误

问题背景 #

使用Node.js 18开发AWS Lambda函数，通过SQS并发查询约15个实体时，出现以下错误：

{"error":"Cannot overwrite `Entity` model once compiled."}

Lambda入口handler代码：

export const handler = async (event) => {
  try {
    const dbClient = await connectToMongoDB();
    const { entityId } = event;
    // Fetch entity data
    const entityData = await fetchEntityData(entityId);
    // further implementations of method
  } catch (error) {
    // 错误处理
    throw error;
  }
}

fetchEntityData方法实现：

const fetchEntityData = async (entityId) => {
  try {
    //declaring model without schema and strict false
    const EntitySchema = new Schema({}, { strict: false });
    const Entity = mongoose.model('Entity', EntitySchema, 'entities');
    // Use the Mongoose model for "Entity" to query the entity based on the entity id  
    const entityData = await Entity.findById(entityId);
    if (entityData) {
      return {
        name: entityData.name,
        location: entityData.location
      };
    } else {
      throw new Error(`Entity with entityID ${entityId} not found.`);
    }
  } catch (error) {
    throw error;
  }
};

错误原因 #

Mongoose的模型一旦编译完成就无法被覆盖重定义。AWS Lambda的并发请求会复用已有的执行容器，当多个请求在同一个容器中执行时，fetchEntityData会重复创建Entity模型，触发该错误。

修复方案 #

修改fetchEntityData方法，创建模型前先检查是否已存在，存在则直接复用：

const fetchEntityData = async (entityId) => {
  try {
    // 优先复用已编译的模型，避免重复创建
    let Entity = mongoose.models.Entity;
    if (!Entity) {
      const EntitySchema = new Schema({}, { strict: false });
      Entity = mongoose.model('Entity', EntitySchema, 'entities');
    }
    // 查询数据
    const entityData = await Entity.findById(entityId);
    if (entityData) {
      return {
        name: entityData.name,
        location: entityData.location
      };
    } else {
      throw new Error(`Entity with entityID ${entityId} not found.`);
    }
  } catch (error) {
    throw error;
  }
};

优化思路 #

1. 全局初始化模型：将模型定义移到Lambda handler之外的全局作用域，利用Lambda容器复用特性，只在容器初始化时创建一次模型：

// 全局作用域定义模型
let Entity;
if (!mongoose.models.Entity) {
  const EntitySchema = new Schema({}, { strict: false });
  Entity = mongoose.model('Entity', EntitySchema, 'entities');
}

const fetchEntityData = async (entityId) => {
  try {
    const entityData = await Entity.findById(entityId);
    if (entityData) {
      return {
        name: entityData.name,
        location: entityData.location
      };
    } else {
      throw new Error(`Entity with entityID ${entityId} not found.`);
    }
  } catch (error) {
    throw error;
  }
};

export const handler = async (event) => {
  try {
    const dbClient = await connectToMongoDB();
    const { entityId } = event;
    const entityData = await fetchEntityData(entityId);
    // further implementations of method
  } catch (error) {
    throw error;
  }
}

2. 复用MongoDB连接：优化connectToMongoDB方法，避免每次请求都创建新连接，利用Lambda容器复用特性维护长连接：

let dbConnection;
const connectToMongoDB = async () => {
  if (dbConnection) {
    return dbConnection;
  }
  dbConnection = await mongoose.connect(process.env.MONGODB_URI);
  return dbConnection;
};

3. 批量查询优化：如果SQS消息是批量发送的，可以在handler中一次性获取所有entityId，执行批量查询（Entity.find({_id: {$in: entityIds}})），减少数据库请求次数，提升并发性能。

内容的提问来源于stack exchange，提问作者Ravikumar Sharma