Aggregation scalability in mongodb

Question

I have been using the aggregate method in Mongodb to make queries, the relationship between other documents or certain transformations is very easy for me. I've gotten to the point where I have to copy various parts of one query into another, especially in $lookups and certain other things. I feel like I'm doing a lot of boilerplate and also when I have to change the behavior of a query I have to make the change in several parts of the code where the same query is made.

Is there a design pattern to improve this? or perhaps it is handled differently? as long as it is scalable and the least amount of code changes have to be made when there is a new requirement. Any resource or help is welcome. Thank you very much, I am new to Mongodb. By the way, I usually use nestjs and express

Answer 1

The MongoDB Aggregation Pipeline is a Turing Complete language, based on function composition. Just don't use it as a big JSON doc, break your aggregation down in parts, like this example:

db.books.aggregate([
    {$match: {year: 1985, pages: 150}}, {$project: {_id: 0, title: 1, year: 1, totalInventory: 1, available: 1}}, {$limit: 15}
])

Should be written as:

let booksFrom1985With150pages = {$match: {year: 1985, pages: 150}};
let showOnlyTheseFields = {$project: {_id: 0, title: 1, year: 1, totalInventory: 1, available: 1}};
let getJust15books =  {$limit: 15};

db.books.aggregate([
    booksFrom1985With150pages,
    showOnlyTheseFields,
    getJust15books,
]);

Which as you can see is more readable, testable, reusable, etc.