mongodb query should be covered by index but is not

Question

the query:

db.myColl.find({"M.ST": "mostrepresentedvalueinthecollection", "M.TS": new Date(2014,2,1)}).explain()

explain output :

"cursor" : "BtreeCursor M.ST_1_M.TS_1",
        "isMultiKey" : false,
        "n" : 587606,
        "nscannedObjects" : 587606,
        "nscanned" : 587606,
        "nscannedObjectsAllPlans" : 587606,
        "nscannedAllPlans" : 587606,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 9992,
        "nChunkSkips" : 0,
        "millis" : 174820,
        "indexBounds" : {
                "M.ST" : [
                        [
                                "mostrepresentedvalueinthecollection",
                                "mostrepresentedvalueinthecollection"
                        ]
                ],
                "M.TS" : [
                        [
                                ISODate("2014-03-01T00:00:00Z"),
                                ISODate("2014-03-01T00:00:00Z")
                        ]
                ]
        },
        "server" : "myServer"

additional details: myColl contains about 40m documents, average object size is 300b.

I don't get why indexOnly is not set to true, I have a compound index on {"M.ST":1, "M.TS":1}

The mongo host is a unix box with 16gb RAM and 500gb disk space (spinning disk). The total index size of the database is 10gb, we've got around 1k upserts/sec, on those 1K 20 are inserts the rest are Increments.

We have another query that adds a third field in the find query (called "M.X"), and also a compound index on "M.ST", "M.X", "M.TS". That one is lightning fast and scans only 330 documents.

Any idea what could be wrong ?

Thanks.

EDIT : here's the structure of a sample document:

{
    "_id" : "somestring",
    "D" : {
        "20140301" : {
            "IM" : {
                "CT" : 143
            }
        },
        "20140302" : {
            "IM" : {
                "CT" : 44
            }
        },
        "20140303" : {
            "IM" : {
                "CT" : 206
            }
        },
        "20140314" : {
            "IM" : {
                "CT" : 5
            }
        }
    },
    "Y" : "someotherstring",
    "IM" : {
        "CT" : 1
    },
    "M" : {
        "X" : 99999,
        "ST" : "mostrepresentedvalueinthecollection",
        "TS" : ISODate("2014-03-01T00:00:00.000Z")
    },
}

The idea is to store some analytics metrics by month, the "D" field represents an array of documents containing data for each day of the month.

Answer 1

EDIT: This feature is not currently implemented. Corresponding JIRA ticket is SERVER-2104. You can upvote for it, but for now, to utilize covered index queries you need to avoid use of dot-notation/embedded document.