Golang byte array communication through channel loses data

Question

I'm using a work-worker Queue implementation in my golang project. The WorkRequest Struct is as below

type WorkRequest struct {
fields map[string] []byte }

and the dipatcher is

go func() {
    for {
        select {
        case work := <-WorkQueue:
            go func() {
                worker := <-WorkerQueue
                worker.Work <- work
            }()
        }
    }
}()

The worker.Work is channel for WorkRequest struct and WorkerQueue is channel of Worker struct. Whenever there are many concurrent requests coming in WorkQueue it gets assigned to worker byt the dispatcher routine.

I'm observing issue when I send concurrent requests to WorkQueue, few of them miss out and few of them get executed multiple times.

e.g. If I've queued works with ID 1, 2, 3, 4, 5... then I see workers getting work with ID 1, 2, 4, 4, 5... here 4 is executed twice and 3 is missed. The same code works fine when I implement WorkRequest fields as map[string]string instead of []byte.

Am I missing something or is slice not thread safe and gets assigned same variable.

Answer 1

In Go string is immutable, so it works fine but all slices are mutable. Moreover they can share the backing array. You are not showing the relevant code but I guess you are expecting the map to hold copy of a slice. A slice is just address in memory and length. You need to manually make a new slice and copy its contents.

To fix your code make a copy of slice when putting it into map.