How to Handle Invalid Mongo/Mongoose ObjectId Assignment in ETL

Question

I have created an ETL to transfer data from sybase to a mongoDB environment. In one place I am matching against a legacy id in order to populate the correct mongo ObjectId in the model.

This works for my first few thousand records, until I run into a record that's missing a legacy id, in which case I get an error because the model is expecting an objectId. Doing a null check in my ETL file isn't sufficient to handle this. I need to also handle the fact that the model is expecting a valid objectId. How would I do this?

My relevant ETL code looks like this:

let agency = await Agency.findOne({ agencyId: record.agent_house }).exec();

Which I drop in like this:

agency: {
   id: agency._id ? agency._id : null,
   // Other prop,
   // Other prop
 }

And the data then gets ported to the model, which looks like this:

agency: {
  id: { type: mongoose.Schema.Types.ObjectId, ref: 'agencies' },
  // other prop,
  // other prop
 }

How can I handle a situation where there is no value, which will cause the objectId assignment to fail even with the null check in place in the ETL file?

Answer 1

Using the mongoose ObjectId type, you can do:

agency: {
   id: agency._id ? agency._id : new mongoose.mongo.ObjectId(),
   // Other prop,
   // Other prop
 }