AWS Lambda Concurrent execution throw Cannot overwrite `Entity` model once compiled

Question

I'm working with a AWS Lambda method using nodejs 18.

The entry point handler method is as below:

export const handler = async (event) => {
  try {

    const dbClient = await connectToMongoDB();

    const { entityId } = event;

    // Fetch entity data
    const entityData = await fetchEntityData(entityId);

    // further implementations of method

  }

And fetchEntityData is as below:

const fetchEntityData = async (entityId) => {
  try {
    //declaring model without schema and strict false
    const EntitySchema = new Schema({}, { strict: false });
    const Entity = mongoose.model('Entity', EntitySchema, 'entities');

    // Use the Mongoose model for "Entity" to query the entity based on the entity id  
    const entityData = await Entity.findById(entityId);

    if (entityData) {
      return {
        name: entityData.name
        location: entityData.location
      };
    } else {
      throw new Error(`Entity with entityID ${entityId} not found.`);
    }
  } catch (error) {
    throw error;
  }
};

This works fine if Lambda executed simulteniously using test button from two different browsers.

However,When Iuse a SQS to run this method concurrently for several entities (~15), I see below error in logs.

{"error":"Cannot overwrite `Entity` model once compiled."}

How can I improve here to prevent this error and achieve concurrent execution. Feel free to guide with other approaches if this is not best optimised way.

Answer 1

The error message "Cannot overwrite Entity model once compiled" is occurring because you are defining the Entity model inside the fetchEntityData function, and it's being called concurrently by multiple Lambda invocations. When multiple Lambda invocations try to define the same Mongoose model at the same time, you can run into this issue.

To improve your setup and allow for concurrent execution of Lambda functions without this error, you should define the Mongoose model outside the fetchEntityData function, preferably at the module level. This way, the model is defined once and can be reused across multiple invocations.

Here's how you can modify your code:

// Define the Mongoose model at the module level
const EntitySchema = new Schema({}, { strict: false });
const Entity = mongoose.model('Entity', EntitySchema, 'entities');

export const handler = async (event) => {
  try {
    const dbClient = await connectToMongoDB();
    const { entityId } = event;

    // Fetch entity data
    const entityData = await fetchEntityData(entityId);

    // further implementations of method
  } catch (error) {
    // Handle errors
  }
};

const fetchEntityData = async (entityId) => {
  try {
    // Use the pre-defined Mongoose model for "Entity" to query the entity based on the entity id
    const entityData = await Entity.findById(entityId);

    if (entityData) {
      return {
        name: entityData.name,
        location: entityData.location
      };
    } else {
      throw new Error(`Entity with entityID ${entityId} not found.`);
    }
  } catch (error) {
    throw error;
  }
};

By defining the Mongoose model at the module level, it will be created only once, and all Lambda invocations can share the same model, avoiding the "Cannot overwrite Entity model once compiled" error. This approach is more efficient and thread-safe when dealing with concurrent Lambda executions.

Additionally, make sure you have proper error handling in your Lambda function to handle any exceptions that might occur during the database operations and return meaningful error responses when needed.