Search code examples
javascriptmongodbmongoosemongoose-schema

Resulting document after update is larger than 16777216

I'm getting two errors when I try adding new data to an array with mongoose. Here is my code:

return await db.fileMeta.findOneAndUpdate({
    username: username,
    'files.fileUID': {
        $ne: data.fileUID
    }
}, {
    $addToSet: {
        files: data
    }
}).exec();

data is a JavaScript object with 25 items that amounts to 789 bytes... Nothing near 16MBs. My code worked fine until recently (last few days) then I started getting this error:

MongoError: BSONObj size: 16829075 (0x100CA93) is invalid. Size must be between 0 and 16793600(16MB) First element: $v: 1
    at MessageStream.messageHandler (/home/user/projects/web-app/node_modules/mongoose/node_modules/mongodb/lib/cmap/connection.js:268:20)
    at MessageStream.emit (events.js:314:20)
    at processIncomingData (/home/user/projects/web-app/node_modules/mongoose/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/home/user/projects/web-app/node_modules/mongoose/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at doWrite (_stream_writable.js:403:12)
    at writeOrBuffer (_stream_writable.js:387:5)
    at MessageStream.Writable.write (_stream_writable.js:318:11)
    at TLSSocket.ondata (_stream_readable.js:719:22)
    at TLSSocket.emit (events.js:314:20)
    at addChunk (_stream_readable.js:298:12)
    at readableAddChunk (_stream_readable.js:273:9)
    at TLSSocket.Readable.push (_stream_readable.js:214:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:188:23) {
  ok: 0,
  code: 10334,
  codeName: 'BSONObjectTooLarge'
}

Then the above error stopped and I got anther instead:

MongoError: Resulting document after update is larger than 16777216
    at MessageStream.messageHandler (/home/user/projects/web-app/node_modules/mongoose/node_modules/mongodb/lib/cmap/connection.js:268:20)
    at MessageStream.emit (events.js:314:20)
    at processIncomingData (/home/user/projects/web-app/node_modules/mongoose/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/home/user/projects/web-app/node_modules/mongoose/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at doWrite (_stream_writable.js:403:12)
    at writeOrBuffer (_stream_writable.js:387:5)
    at MessageStream.Writable.write (_stream_writable.js:318:11)
    at TLSSocket.ondata (_stream_readable.js:719:22)
    at TLSSocket.emit (events.js:314:20)
    at addChunk (_stream_readable.js:298:12)
    at readableAddChunk (_stream_readable.js:273:9)
    at TLSSocket.Readable.push (_stream_readable.js:214:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:188:23) {
  ok: 0,
  code: 17419,
  codeName: 'Location17419'
}

Using MongoDB Compass I can see this:

enter image description here

Note the total size of 16.2MB, this is the only thing that I can think of that is even lose to 16MBs.

I would understand that if the error was trying to say that my data object was too large, but since it is so small (789 bytes) I don't understand why I'm getting the error or how to fix it. If the error is because the entire DB is larger than 16MBs then something must be wrong because obviously DBs should scale larger than 16MBs.

How can I prevent this error?

Solution

  • I took @Evert's advice and flattened my DB so that each file's metadata was it's own document instead of an array within a document. And now it works fine.

    const fileMetadataDocument = new db.FileMetadata(data);

const saved = await fileMetadataDocument.save();
//Works