Search code examples
node.jsmongodbmongooseinsertinsert-update

How to insertMany and update duplicates in mongodb?


I have many orders I would like to insert in MongoDB, the orders are unique indexed by orderID.

I would like to insert all the orders and update order time if one of my order was exists in the database.

I have tried to use updateMany with upsert set to true, but it didn't work as expected.

static async addOrders(ordersArray) {
    const ordersTable = new mongoose.model('ordersTable', Schemas.ordersTable);

    try {
        const docs = await ordersTable.insertMany(ordersArray, {ordered: false});
        return Promise.resolve('Data Inserted');
    } catch (e) {
        return Promise.reject(e);
    }
}

Solution

  • You can find duplicates as follows, then you can remove them and insert them again.

    example:

    static async addOrders(ordersArray) {
    const ordersTable = new mongoose.model('ordersTable', Schemas.ordersTable);
    
    try {
        const docs = await ordersTable.insertMany(ordersArray, {ordered: false});
        return Promise.resolve('Data Inserted');
    } catch (e) {
           if (e.code === 11000){
                // 1- getting duplicates
                console.log('getting duplicates');
    
                let orders = [];
                const ordersIDs = e.result.result.writeErrors.map(error=>{
                    const parsedError = JSON.stringify(error);
                    const order = JSON.parse(parsedError).op;
                    orders.push(order);
                    return order.orderID;
                });
                // 2- removing old duplicates.
                const deleted = await ordersTable.deleteMany({orderID:{'$in':ordersIDs}});
                // 3- adding the orders
                try{
    
                    const newAddedOrders = await ordersTable.insertMany(orders , {ordered : false});
                    return Promise.resolve('Data Inserted');
                }catch (e) {
                    return Promise.reject(e);
                }
    
    
            }else return Promise.reject(e);
    
    }
    

    }

    As you see in the code comments, we are getting duplicates first, then we are removing and inserting the rows.