Why doesn't the goroutines process the same file twice?

Question

I'm looking at the goroutines blog post about patterns with goroutines and channels etc.

In the bounded.go example I see this:

    paths, errc := walkFiles(done, root)

    // Start a fixed number of goroutines to read and digest files.
    c := make(chan result) // HLc
    var wg sync.WaitGroup
    const numDigesters = 20
    wg.Add(numDigesters)
    for i := 0; i < numDigesters; i++ {
        go func() {
            digester(done, paths, c) // HLc
            wg.Done()
        }()
    }

Now since each digester is processing the same paths collection, why doesn't it repeat the same file twice?

Answer 1

walkFiles, which was not reproduced in your question but which is key to understanding it, has the following signature:

func walkFiles(done <-chan struct{}, root string) (<-chan string, <-chan error)

So in your quoted code, paths isn't a "collection" (i.e. a slice), it's a channel. When each of the workers reads from the channel, it pulls the next path out of the channel. The next worker to receieve from the channel won't get the same path, it will get the next one after that.

All three of the arguments to digester are channels:

func digester(done <-chan struct{}, paths <-chan string, c chan<- result) 

• done is used to indicate to the workers that they should stop, even if there is still work queued.
• paths is the work queue the workers receive paths from.
• c is the channel that the workers send results on.