How can I retrieve large number of documents from Mongo datastore using Morphia?

Question

I am using following code to iterate through a database's documents.

public void readDataStore() throws IOException {
        Query query = document_datastore.find(DocumentPojo.class);
        List<DocumentPojo> documentPojos = query.asList();

        documentPojos.forEach(obj -> {
                    try {
                        System.out.println(obj.getDocid());
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
        );    
}

Currently the DB has not more than 100 documents but in future it can have ~100000 documents. I suspect then it may run into performance issues?

DocumentPojo is the class that I map the results to. I am using Java 8 and Morphia.

How should I solve this issue?

Answer 1

Use query.fetch() to get the MorphiaIterator and then handle each document as you get it. It won't pull them all in to memory at once allowing you to process your hundred thousand+ documents.