MongoDb: Pipeline inside lookup error: Error: Arguments must be aggregate pipeline operators

Question

I have these three models:

1. Institution model:

const InstitutionSchema = new Schema({
  current_students: [
    {
      type: Schema.Types.ObjectId,
      ref: "users",
    },
  ],
});

module.exports = Institution = mongoose.model("institutions", InstitutionSchema);

which has array of references to User model in current_students field.

2. User model:

const UserSchema = new Schema({
  profile: {
    type: Schema.Types.ObjectId,
    ref: "profiles",
  },
});

module.exports = User = mongoose.model("users", UserSchema);

which has references to Profile model in profile field.

3. Profile model:

const ProfileSchema = new Schema({
  videoURL: {
    type: String,
  },
});

module.exports = Profile = mongoose.model("profiles", ProfileSchema);

I am trying to get the list of profiles for users of an institution where Profile.videoURL is not null or undefined. This is what I have tried:

Institution.aggregate([
  {
    $lookup: {
      from: "users",
      localField: "current_students",
      foreignField: "_id",
      as: "current_student",
    },
  },
  {
    $unwind: {
      path: "$current_student",
      preserveNullAndEmptyArrays: true,
    },
  },
  {
    $lookup: {
      from: "profiles",
      localField: "current_student.profile",
      foreignField: "_id",
      as: "current_student_profile",
    },
    pipeline: [
      {
        $match: {
          videoURL: { $nin: [undefined, null] },
        },
      },
    ],
  },
]);

However, I keep getting this error because of the last pipeline that executes the $match operation.

Error: Arguments must be aggregate pipeline operators

Any idea how to fix this?

Answer 1

Your query is wrong. You cannot pass pipeline like that. Have a look at the $lookup syntax. Also if you want to learn more about aggregations, I would suggest you to take up this Aggregation course offered by MongoDB themselves. It free for all.

Try this query:

db.institutions.aggregate([
    {
        $lookup: {
            from: "users",
            localField: "current_students",
            foreignField: "_id",
            as: "current_student",
        }
    },
    {
        $unwind: {
            path: "$current_student",
            preserveNullAndEmptyArrays: true,
        }
    },
    {
        $lookup: {
            from: "profiles",
            localField: "current_student.profile",
            foreignField: "_id",
            as: "current_student_profile",
        }
    },
    { $unwind: "$current_student_profile" },
    {
        $match: {
            "current_student_profile.videoURL": { $nin: [undefined, null] },
        }
    }
]);

Output:

/* 1 createdAt:3/13/2021, 6:18:26 PM*/
{
    "_id" : ObjectId("604cb49a6b2dcb17e8b152b2"),
    "name" : "Institute 1",
    "current_students" : [
        ObjectId("604cb4c36b2dcb17e8b152b8"),
        ObjectId("604cb4c36b2dcb17e8b152b9")
    ],
    "current_student" : {
        "_id" : ObjectId("604cb4c36b2dcb17e8b152b8"),
        "name" : "Dheemanth Bhat",
        "profile" : ObjectId("604cb4b16b2dcb17e8b152b5")
    },
    "current_student_profile" : {
        "_id" : ObjectId("604cb4b16b2dcb17e8b152b5"),
        "videoURL" : "http://[email protected]"
    }
},

/* 2 createdAt:3/13/2021, 6:18:26 PM*/
{
    "_id" : ObjectId("604cb49a6b2dcb17e8b152b3"),
    "name" : "Institute 2",
    "current_students" : [
        ObjectId("604cb4c36b2dcb17e8b152ba")
    ],
    "current_student" : {
        "_id" : ObjectId("604cb4c36b2dcb17e8b152ba"),
        "name" : "Alex Rider",
        "profile" : ObjectId("604cb4b16b2dcb17e8b152b7")
    },
    "current_student_profile" : {
        "_id" : ObjectId("604cb4b16b2dcb17e8b152b7"),
        "videoURL" : "http://[email protected]"
    }
}

Test data:

users collection:

/* 1 createdAt:3/13/2021, 6:19:07 PM*/
{
    "_id" : ObjectId("604cb4c36b2dcb17e8b152b8"),
    "name" : "Dheemanth Bhat",
    "profile" : ObjectId("604cb4b16b2dcb17e8b152b5")
},

/* 2 createdAt:3/13/2021, 6:19:07 PM*/
{
    "_id" : ObjectId("604cb4c36b2dcb17e8b152b9"),
    "name" : "Ahmed Ghrib",
    "profile" : ObjectId("604cb4b16b2dcb17e8b152b6")
},

/* 3 createdAt:3/13/2021, 6:19:07 PM*/
{
    "_id" : ObjectId("604cb4c36b2dcb17e8b152ba"),
    "name" : "Alex Rider",
    "profile" : ObjectId("604cb4b16b2dcb17e8b152b7")
}

profiles collection:

/* 1 createdAt:3/13/2021, 6:18:49 PM*/
{
    "_id" : ObjectId("604cb4b16b2dcb17e8b152b5"),
    "videoURL" : "http://[email protected]"
},

/* 2 createdAt:3/13/2021, 6:18:49 PM*/
{
    "_id" : ObjectId("604cb4b16b2dcb17e8b152b6")
},

/* 3 createdAt:3/13/2021, 6:18:49 PM*/
{
    "_id" : ObjectId("604cb4b16b2dcb17e8b152b7"),
    "videoURL" : "http://[email protected]"
}

institutions collection

/* 1 createdAt:3/13/2021, 6:18:26 PM*/
{
    "_id" : ObjectId("604cb49a6b2dcb17e8b152b2"),
    "name" : "Institute 1",
    "current_students" : [
        ObjectId("604cb4c36b2dcb17e8b152b8"),
        ObjectId("604cb4c36b2dcb17e8b152b9")
    ]
},

/* 2 createdAt:3/13/2021, 6:18:26 PM*/
{
    "_id" : ObjectId("604cb49a6b2dcb17e8b152b3"),
    "name" : "Institute 2",
    "current_students" : [
        ObjectId("604cb4c36b2dcb17e8b152ba")
    ]
}