What is the best way to process a large list of Domain Objects?

Question

What is the best way to process a large list of Domain Objects? For example, I have 'Users' and 'Book' domains, and there is a permission READ on Book object. When I'm adding a new Book, I'd like to set READ permission to all users for this Book. At first was a code:

def users = Users.findAll{ ... }
users.each { addPermission(book, it, READ) }  

I'm using Spring Security Core and ACL plugin. But now, I think it is not best way to load 10000 Users Objects to memory. I gonna use the SCROLL method with maxResults(???) from Criteria. So my question is What the best way? How to determinate the best number of MaxResults?

Answer 1

A more of Grails way to do this would be to use the batch processing. Try the example given below: EDIT : Improved answer. Now, using pagination based batch processing.

             def noOfObjectsTobeProcessedAtAtime=1000//Step or pagination size...

    List offsetMaxMapList = (0..User.count()-1).step(noOfObjectsTobeProcessedAtAtime).collect{[max:noOfObjectsTobeProcessedAtAtime,offset:it]}
    offsetMaxMapList.each{offsetMaxMap->
        addPermissionToUserInBatch(params)  

    }

    def addPermissionToUserInBatch(params){
        def batch = []
                def session
                def users = Users.createCriteria().list(params){}

                users.eachWithIndex { user, index ->
                    batch << user

                    if (batch.size() >= batchSize) {
                        User.withTransaction {
                            batch.each {User userObject ->
                             addPermission(book, userObject, READ)
                                }
                        }
                        batch.clear()

                    } else if (batch.size() < batchSize && (users.size() - index - 1) == 0) {
                        User.withTransaction {
                            batch.each {User userObject ->
                                 addPermission(book, userObject, READ)
                                }
                        }
                        batch.clear()
                    }
                    session = sessionFactory.getCurrentSession()
                    session.clear()
                }

    }

Hope that helps!!!