Using Goroutines for background work inside an http handler

I thought I'd found an easy way to return an http response immediately then do some work in the background without blocking. However, this doesn't work.

func MyHandler(w http.ResponseWriter, r *http.Request) {
    //handle form values
    go doSomeBackgroundWork() // this will take 2 or 3 seconds
    w.WriteHeader(http.StatusOK)
}

It works the first time--the response is returned immediately and the background work starts. However, any further requests hang until the background goroutine completes. Is there a better way to do this, that doesn't involve setting up a message queue and a separate background process.

Solution

Go multiplexes goroutines onto the available threads which is determined by the GOMAXPROCS environment setting. As a result if this is set to 1 then a single goroutine can hog the single thread Go has available to it until it yields control back to the Go runtime. More than likely doSomeBackgroundWork is hogging all the time on a single thread which is preventing the http handler from getting scheduled.

There are a number of ways to fix this.

First, as a general rule when using goroutines, you should set GOMAXPROCS to the number of CPUs your system has or to whichever is bigger.

Second, you can yield control in a goroutine by doing any of the following:

runtime.Gosched()
ch <- foo
foo := <-ch
select { ... }
mutex.Lock()
mutex.Unlock()

All of these will yield back to the Go runtime scheduler giving other goroutines a chance to work.