How do you keep goroutines running if another one crashes?

Question

I've been trying to find something that answers this question but I can't find anything that talks about it.

Lets say I have a function in Go which is something like this:

func main() {
    // assume this wrapped in a waitgroup or something 
    // so that it doesnt exit
    go queue.ConsumeAndDoSomething()
    go api.StartServer()
}

I have two goroutines here that do completely different things and one should ideally keep running if the other crashes/panics. If the queue operation fails, the API server should be impacted and vice versa.

I'm not sure if this possible (or even recommended). Is there a clean way of doing this or should the whole program exit once a goroutine panics?

Answer 1

You have to use the builtin recover() function to recover from panics, and you have to call it in a deferred function.

Let's say you have a function that may panic:

func doPanic() {
    log.Println("about to panic")
    panic("test")
}

Create a helper function to launch a function as a goroutine "protected" (from panics):

func protect(f func()) {
    defer func() {
        if err := recover(); err != nil {
            log.Printf("Recovered: %v", err)
        }
    }()

    f()
}

And use it like this:

func main() {
    go protect(doPanic)

    for {
        time.Sleep(time.Second)
        fmt.Println("tick")
    }
}

This test app will output:

2021/03/04 14:12:31 about to panic
2021/03/04 14:12:31 Recovered: test
tick
tick
tick
...

See related question: Generic panic recovering in go programs