Search code examples
gomgo

fatal error out of memory

So I wrote a daemon in go that handles around 800k documents and i'm having an out of memory problem.

From what i saw when getting the documents from mongodb the memory usage increases with every loop.

func main() {
session, err := mgo.Dial("localhost")
if err != nil { panic(err) }
defer session.Close()

subscriptionsC  = session.DB("sm").C("subscriptions")
subscriptions   := []Subscription{}

for {
    subscriptions = GetSubscriptions()

And the other function is:

func GetSubscriptions()([]Subscription) {
    result  := []Subscription{}
    err    := subscriptionsC.Find(nil).Prefetch(0.0).All(&result)

    if err != nil { Log("signups_err", err.Error() + "\n") }
    return result
}

I don't know if it's redeclaring the array with each loop or what exactly happens.

Any help would be greatly appreciated.

Solution

  • The array is definietly being inialized in every loop because of the call to GetSubscriptions() and then inside the loop result := []Subscription{}, but I think that's not the source of the problem.

    The problem could be coming from your global session, see Database connections in web applications, The proper way would be by using a session pool.

    Edit: also see How do I call mongoDB CRUD method from handler?