Mongodb cursor.toArray() is too slow

Question

I am using cursor.toArray() to return my collection.find(query) as a list and response time for my API is in 100's of milliseconds. Data fetched into the cursor is very less (a couple of hundred records), the database is indexed on the field I am querying. I have also set the batch size cursor.batchSize(1000).

db.collection.find({"{ "ZIP" : { "$in" : [ "12345"]}}"}) is my query, and my databse is indexed on 'ZIP' . I can see the same query running on the shell within 4 ms.

The same query on mongo shell hardly takes 5 ms.

Mogo driver I am using is :

<!-- https://mvnrepository.com/artifact/org.mongojack/mongojack -->
<dependency>
    <groupId>org.mongojack</groupId>
    <artifactId>mongojack</artifactId>
    <version>2.8.2</version>
</dependency>

The code

@Path("/")
@Produces(MediaType.APPLICATION_JSON)
@Api(value = "listing-mongo")
public class MlsMongoResource {

    private JacksonDBCollection<Mlsdatadao, String> collection;
    Clock clock;

    public MlsMongoResource(JacksonDBCollection<Mlsdatadao, String> collection) {
        this.collection = collection;
        this.clock =  Clock.systemUTC();
    }

    @GET
    @Path("/listings-mongo")
    @Produces(value = MediaType.APPLICATION_JSON)
    @Timed
    public List<Mlsdatadao> getListings(@BeanParam MlsListingParameters mlsBeanParam) {

        BasicDBList basicDbList = new BasicDBList();
        mlsBeanParam.validateBean();

        setLocations(basicDbList,mlsBeanParam.zipcodes);

        BasicDBObject query = new BasicDBObject("$and", basicDbList);

        DBCursor<Mlsdatadao> cursor = null;

        long start = 0;
        try{
             start = System.currentTimeMillis();
            cursor = collection.find(query);
            cursor.batchSize(1000);

        } catch (Exception e){
            System.out.println("IN collection.find() " +  e.getCause());
        }


        System.out.println("QUERY LIST IS " + basicDbList);

        if(cursor == null) {
            System.out.println("Cursor is null");
        }
        List<Mlsdatadao> result = cursor.toArray();
       cursor.close();(System.currentTimeMillis() - start));
        return result;

    }



    private void setLocations(BasicDBList basicDbList, List<String> zipcodes) {

        if (CollectionUtils.isNotEmpty(zipcodes)) {
            basicDbList.add(setZipcodes(zipcodes));
        }

    }

    private BasicDBObject setZipcodes(List<String> zipcodes) {
        return new BasicDBObject("ZIP" ,  new BasicDBObject("$in", zipcodes) );
    }
}

Application:

public class MongoApplication extends Application <MlsMongoConfiguration> {

    public static void main(String[] args) throws Exception {
        new MlsMongoApplication().run(args);
    }

    @Override
    public String getName() {
        return "mls-dropwizard-mongo";
    }

    @Override
    public void initialize(Bootstrap<MlsMongoConfiguration> bootstrap) {

        bootstrap.addBundle(new SwaggerBundle<MlsMongoConfiguration>() {
            @Override
            protected SwaggerBundleConfiguration getSwaggerBundleConfiguration(MlsMongoConfiguration configuration) {
                return configuration.swaggerBundleConfiguration;
            }
        });

    }

    @Override
    public void run(MlsMongoConfiguration configuration, Environment environment) throws Exception {


        MongoClientOptions.Builder clientOptions = new MongoClientOptions.Builder();
        clientOptions.minConnectionsPerHost(1000);//min
        clientOptions.maxWaitTime(1000);
        clientOptions.connectionsPerHost(1000);

        //Create Mongo instance
        //Mongo mongo = new Mongo(configuration.mongohost, configuration.mongoport);

        MongoClient mongoClient = new MongoClient(new ServerAddress(configuration.mongohost, configuration.mongoport), clientOptions.build());

        //Add Managed for managing the Mongo instance
        //MongoManaged mongoManaged = new MongoManaged(mongo);
        MongoManaged mongoManaged = new MongoManaged(mongoClient);
        environment.lifecycle().manage(mongoManaged);

        //Add Health check for Mongo instance. This will be used from the Health check admin page
        environment.healthChecks().register("MongoHealthCheck", new MongoHealthCheck(mongoClient));
        //Create DB instance and wrap it in a Jackson DB collection
        DB db = mongoClient.getDB(configuration.mongodb);
        JacksonDBCollection<Mlsdatadao, String> jacksonDBCollection = JacksonDBCollection.wrap(db.getCollection("mlsdata"), Mlsdatadao.class, String.class);
        environment.jersey().register(new MlsMongoResource(jacksonDBCollection));
    }
}

Is there any way to avoid cursor.toArray()? Any other performance tuning hints would be really helpful.

Thanks.

Answer 1

Things look good after changing my MongoDB driver from mongojack to native mongo-java-driver 3.7 and used com.mongodb.client.FindIterable instead of DBCursor. Looks like Mongojack library is spending a lot of time in mapping BSON objects to POJOs.