Search code examples
gobatch-processingchannel

Reading multiple elements from a channel in Go


I'm reading values from a channel in a loop like this:

for {
    capturedFrame := <-capturedFrameChan
    remoteCopy(capturedFrame)
}

To make it more efficient, I would like to read these values in a batch, with something like this (pseudo-code):

for {
    capturedFrames := <-capturedFrameChan
    multipleRemoteCopy(capturedFrames)
}

But I'm not sure how to do that. If I call capturedFrames := <-capturedFrameChan multiple times it's going to block.

Basically, what I would like is to read all the available values in captureFrameChan and, if none is available, it blocks as usual.

What would be the way to accomplish this in Go?


Solution

  • I've ended up doing it as below. Basically I've used len(capturedFrames) to know how many frames are available, then retrieved them in a loop:

    for {
        var paths []string
        itemCount := len(capturedFrames)
        if itemCount <= 0 {
            time.Sleep(50 * time.Millisecond)
            continue
        }
    
        for i := 0; i < itemCount; i++ {
            f := <-capturedFrames
            paths = append(paths, f)
        }
    
        err := multipleRemoteCopy(paths, opts)
        if err != nil {
            fmt.Printf("Error: could not remote copy \"%s\": %s", paths, err)
        }
    }