NodeJS: JavaScript heap out of memory

Question

In my Node.js application, there's a function that causes the application to crash with the error: Reached heap limit Allocation failed - JavaScript heap out of memory. I suspect this could be related to executing a large number of asynchronous operations simultaneously, which may lead to memory exhaustion.

However, I'm not sure how to fix it. Perhaps someone has some ideas?

async updateLinks() {
  const medias = await this.mediaModel.find()
  this.logger.log('Cron Job: Updating Urls .....')
  for(let media of medias) {
    const url = await this.testService.getPreSignedUrl(media.key)
    media.url = url
    media.save()
  }
  this.logger.log('Cron Job: Completed ')
}

async getPreSignedUrl(key: string): Promise<string> {
  try {
    const data = await this.s3.getSignedUrl('getObject', {
      Key: key,
      Bucket: process.env.BUCKET_ID,
      Expires: 600000 // almost a week
    });
    return data;
  } catch (err) {
    this.logger.error(err);
    return null;
  }
}

Answer 1

To keep all your operations from running parallel, add an await in front of media.save() like this:

async updateLinks() {
  const medias = await this.mediaModel.find()
  this.logger.log('Cron Job: Updating Urls .....')
  for(let media of medias) {
    const url = await this.testService.getPreSignedUrl(media.key)
    media.url = url
    await media.save()
  }
  this.logger.log('Cron Job: Completed ')
}

Without that await, updateLinks() returns BEFORE the save is done and thus you get lots of save() operation in flight at the same time which could (if the for loop is large) overwhelm some sort of resource. Instead, with the await, the operations will be sequenced one at a time.

If you were calling updateLinks() more than once or in a loop, you would also want an await on that call.