What's the ideal size of buffered channel and number of workers?

Question

I'm trying to build an asynchronous codec. I have implemented a job dispatcher that has access to a buffered channel of jobs

var JobChannel chan Job = make(chan Job, 100000)

the dispatcher takes as input the number of workers and assigns work to them

func StartDispacher(numberOfWorkers int){
    // start workers
    wg := &sync.WaitGroup{}
    wg.Add(numberOfWorkers)
    for i := int(1); i <= numberOfWorkers; i++ {
        go func(i int) {
            defer wg.Done()
            for j := range JobChannel {
                doWork(i, j)
            }
        }(i)
    }
}

my main function starts the dispatcher and keeps giving it jobs to do (in this case 200000 jobs)

workDispatcher.StartDispacher(2*runtime.NumCPU())
for i := 0; i < 200000; i++ {
    j := workDispatcher.Job{
        BytePacket: d,
        JobType:    workDispatcher.DECODE_JOB,
    }
    workDispatcher.JobChannel <- j
}

after experimenting: turns out there are 2 factors that affect the performance of this code

• the size of the buffered channel JobChannel
• the number of workers there are func StartDispacher(numberOfWorkers int)

Is there a standard way to find the optimal values for these parameters, and is it possible to make these values independent from the physical set-up of the machine running the code?

Answer 1

You always need to measure to determine how the system will perform under load. The good news here is that you only have 2 variables, which are mostly independent, so it's fairly easy to reason about.

The number of workers determines your concurrency, so benchmark the processing to see what the optimal concurrency is. There is usually a number of concurrent processes above which the returns drop off dramatically.

The size of the channel is just like any other "buffer" in a system. A larger buffer can handle larger spikes in input, at the expense of possibly inducing larger latencies and memory usage.