Currently the project is using AEM 6.0 with mongo 2.6.10. Because of an known issue about maxPasses assertion, mongo fails to allocate the space required.
Adobe official doc mentioned that crx storage can be defined to use another file system under certain conditions. In this case, it is required to store the dam assets with size > 16M on local file storage instead of in mongoDB. See repository set up with repository.xml. However, details of how is not specified.
The question is how to config repository.xml to use local file system instead of mongoDB for files larger than a specific size?
I think you have not configured a separate DataStore. That's the reason your larger files are being persisted in your MongoDB. AEM allows you to configure NodeStore and DataStore separately.
Once you configure a separate DataStore, all your larger files(default > 16MB) will be stored separately in DataStore while all your regular nodes & properties will be stored in NodeStore.
There are multiple options to choose NodeStore & DataStores. In your case, I would suggest you to continue using your MongoDB as your NodeStore while configuring a separate FileSystem DataStore to store the binaries.
Please check the below documentation on how to configure separate DataStore :
https://docs.adobe.com/docs/en/aem/6-1/deploy/platform/data-store-config.html
For 6.0 version,
https://docs.adobe.com/docs/en/aem/6-0/deploy/upgrade/data-store-config.html