Search code examples
phpmongodbcursordatabase

PHP Mongo cursor over a subset of data in a document?

I have a Mongo document that contains a large amount of data inside one of its fields, for example achievements in the following document:

{'id':12345,'name':'eric','achievements':['...','...','...']}

Is there any way I can return the achievements field in PHP by itself with a cursor, rather than loading the whole field into memory? When this field is very large (more than one million items) it uses a lot of memory.

I've tried the ->find(['id'=>new MongoId('12345')],['achievements']) syntax, however, the entire achievements field is loaded into memory as an array. Ideally, I want a cursor to load one achievement at a time. Is this even possible? Seems like a pretty cool feature to have...

Cheers

Solution

  • No, MongoDB will return full documents by default. As you suggest, you can use field selection to limit the fields that are returned, and in the case of arrays, you can use the $slice operator to retrieve only a subset of the array. However, when doing so, getting the "next" slice of the array requires a new query and full network round-trip.

    Two other notes:

    1. In MongoDB 2.1 (and the coming 2.2 stable release), you can use the $unwind operator in the Aggregation Framework to emulate this behavior; however, note that in 2.2, the Aggregation Framework will always return results as a single document with a nested array, so this still won't help you avoid the memory issues.

    2. I would strongly consider changing your data model here. An array of "millions of elements" will lead to large documents (MongoDB has a 16MB upper limit on document size) and added overhead in your application code, as you have seen. For some alternate ideas on how to model this, check out Schema Design at Scale which covers data model use cases very much like what you've described here.