I am studying about golang and use of goroutine.
With this sample code, on my PC with 4 logical CPUs, I can't understand why I don't have any performance improvement.
If I add in concurrency, the time it takes to do tends to slow down by the original without concurrency.
2017/10/27 13:57:17 Starting 1 threads
2017/10/27 13:57:27 Id routine: 0 rate: 232.558140
2017/10/27 13:57:27 Current total rate was 232.56 K/s
2017/10/27 13:57:27 Starting 1 threads
2017/10/27 13:57:37 Id routine: 0 rate: 357.142857
2017/10/27 13:57:37 Current total rate was 357.14 K/s
2017/10/27 13:58:04 Starting 2 threads
2017/10/27 13:58:14 Id routine: 0 rate: 131.578947
2017/10/27 13:58:14 Id routine: 1 rate: 129.870130
2017/10/27 13:58:14 Current total rate was 261.45 K/s
2017/10/27 13:58:14 Starting 2 threads
2017/10/27 13:58:24 Id routine: 1 rate: 144.927536
2017/10/27 13:58:24 Id routine: 0 rate: 151.515152
2017/10/27 13:58:24 Current total rate was 296.44 K/s
2017/10/27 13:59:15 Starting 3 threads
2017/10/27 13:59:25 Id routine: 2 rate: 94.339623
2017/10/27 13:59:25 Id routine: 0 rate: 92.592593
2017/10/27 13:59:25 Id routine: 1 rate: 90.090090
2017/10/27 13:59:25 Current total rate was 277.02 K/s
2017/10/27 13:59:25 Starting 3 threads
2017/10/27 13:59:35 Id routine: 2 rate: 92.592593
2017/10/27 13:59:35 Id routine: 0 rate: 90.090090
2017/10/27 13:59:35 Id routine: 1 rate: 101.010101
2017/10/27 13:59:35 Current total rate was 283.69 K/s
2017/10/27 14:00:12 Starting 4 threads
2017/10/27 14:00:22 Id routine: 1 rate: 56.818182
2017/10/27 14:00:22 Id routine: 3 rate: 56.818182
2017/10/27 14:00:22 Id routine: 0 rate: 61.349693
2017/10/27 14:00:22 Id routine: 2 rate: 62.500000
2017/10/27 14:00:22 Current total rate was 237.49 K/s
2017/10/27 14:00:22 Starting 4 threads
2017/10/27 14:00:32 Id routine: 1 rate: 67.114094
2017/10/27 14:00:32 Id routine: 3 rate: 61.349693
2017/10/27 14:00:32 Id routine: 0 rate: 57.803468
2017/10/27 14:00:32 Id routine: 2 rate: 65.789474
2017/10/27 14:00:32 Current total rate was 252.06 K/s
And, here is the concurrent code with the goroutines:
package main
import (
"fmt"
"log"
"sync"
"time"
)
var (
tryArr []string
goroutineQuit = make(chan struct{})
concurrencyMapHQ = struct {
sync.RWMutex
HQ map[int]float64
}{HQ: make(map[int]float64)}
)
func rateStats() {
var totalRate float64
for i, rate := range concurrencyMapHQ.HQ {
log.Printf("Id routine: %d rate: %f", i, rate)
totalRate += rate
}
log.Printf("Current total rate was %.2f K/s", totalRate)
}
func stopThreads(threads int) {
for i := 0; i < threads; i++ {
goroutineQuit <- struct{}{}
}
rateStats()
}
func runThreads(threads int, steps int) {
log.Printf("Starting %d threads", threads)
for i := 0; i < threads; i++ {
i := i
go func(id int, steps int) {
calculate(i, steps)
}(i, steps)
}
}
func calculate(idRoutine int, steps int) {
var (
possibles []string
numBatchs = 0
t1 = time.Now()
)
for {
select {
case <-goroutineQuit:
return
default:
possibles = nil
for _, nonce := range tryArr {
possibles = append(possibles, nonce)
}
numBatchs++
if numBatchs == 99 {
t2 := time.Now()
timeExecution := t2.Sub(t1) / time.Millisecond
concurrencyMapHQ.Lock()
concurrencyMapHQ.HQ[idRoutine] = float64(steps) / float64(timeExecution)
concurrencyMapHQ.Unlock()
numBatchs = 0
t1 = t2
}
}
}
}
func main() {
threads := 4
steps := 10000
tryArr = make([]string, steps)
for i := 0; i < steps; i++ {
tryArr[i] = fmt.Sprintf("indice: %d", i)
}
for {
runThreads(threads, steps)
time.Sleep(10 * time.Second)
stopThreads(threads)
runThreads(threads, steps)
time.Sleep(10 * time.Second)
stopThreads(threads)
}
}
Allocate possibles for the default case:
default:
//possibles = nil
possibles = make([]string, 0, len(tryArr))
For example, possibles = nil,
2017/10/27 10:59:35 GOMAXPROCS: 4
2017/10/27 10:59:35 Starting 4 threads
2017/10/27 10:59:45 Id routine: 3 rate: 20.576132
2017/10/27 10:59:45 Id routine: 2 rate: 21.276596
2017/10/27 10:59:45 Id routine: 0 rate: 19.960080
2017/10/27 10:59:45 Id routine: 1 rate: 21.276596
2017/10/27 10:59:45 Current total rate was 83.09 K/s
2017/10/27 10:59:45 Starting 4 threads
2017/10/27 10:59:55 Id routine: 2 rate: 20.533881
2017/10/27 10:59:55 Id routine: 0 rate: 20.576132
2017/10/27 10:59:55 Id routine: 1 rate: 19.801980
2017/10/27 10:59:55 Id routine: 3 rate: 20.746888
2017/10/27 10:59:55 Current total rate was 81.66 K/s
57.08user 2.16system 0:20.02elapsed 295%CPU
And, possibles = make([]string, 0, len(tryArr)),
2017/10/27 10:57:33 GOMAXPROCS: 4
2017/10/27 10:57:33 Starting 4 threads
2017/10/27 10:57:43 Id routine: 0 rate: 112.359551
2017/10/27 10:57:43 Id routine: 1 rate: 104.166667
2017/10/27 10:57:43 Id routine: 2 rate: 91.743119
2017/10/27 10:57:43 Id routine: 3 rate: 112.359551
2017/10/27 10:57:43 Current total rate was 420.63 K/s
2017/10/27 10:57:43 Starting 4 threads
2017/10/27 10:57:53 Id routine: 0 rate: 123.456790
2017/10/27 10:57:53 Id routine: 1 rate: 126.582278
2017/10/27 10:57:53 Id routine: 2 rate: 113.636364
2017/10/27 10:57:53 Id routine: 3 rate: 119.047619
2017/10/27 10:57:53 Current total rate was 482.72 K/s
58.11user 2.32system 0:20.01elapsed 302%CPU
What are your results?
With increasing "threads" [1,2,3,4] and possibles = make([]string, 0, len(tryArr)),
$ cat /proc/cpuinfo | grep 'cpu cores' | uniq
cpu cores : 4
.
2017/10/27 11:19:03 GOMAXPROCS: 4
2017/10/27 11:19:03 Starting 1 threads
2017/10/27 11:19:13 Id routine: 0 rate: 285.714286
2017/10/27 11:19:13 Current total rate was 285.71 K/s
2017/10/27 11:19:13 Starting 1 threads
2017/10/27 11:19:23 Id routine: 0 rate: 312.500000
2017/10/27 11:19:23 Current total rate was 312.50 K/s
24.66user 1.13system 0:20.01elapsed 128%CPU
.
2017/10/27 11:19:28 GOMAXPROCS: 4
2017/10/27 11:19:28 Starting 2 threads
2017/10/27 11:19:38 Id routine: 0 rate: 238.095238
2017/10/27 11:19:38 Id routine: 1 rate: 217.391304
2017/10/27 11:19:38 Current total rate was 455.49 K/s
2017/10/27 11:19:38 Starting 2 threads
2017/10/27 11:19:48 Id routine: 1 rate: 192.307692
2017/10/27 11:19:48 Id routine: 0 rate: 217.391304
2017/10/27 11:19:48 Current total rate was 409.70 K/s
46.14user 2.07system 0:20.00elapsed 240%CPU
.
2017/10/27 11:19:53 GOMAXPROCS: 4
2017/10/27 11:19:53 Starting 3 threads
2017/10/27 11:20:03 Id routine: 2 rate: 142.857143
2017/10/27 11:20:03 Id routine: 1 rate: 149.253731
2017/10/27 11:20:03 Id routine: 0 rate: 147.058824
2017/10/27 11:20:03 Current total rate was 439.17 K/s
2017/10/27 11:20:03 Starting 3 threads
2017/10/27 11:20:13 Id routine: 2 rate: 153.846154
2017/10/27 11:20:13 Id routine: 1 rate: 156.250000
2017/10/27 11:20:13 Id routine: 0 rate: 153.846154
2017/10/27 11:20:13 Current total rate was 463.94 K/s
52.41user 2.21system 0:20.01elapsed 273%CPU
.
2017/10/27 11:20:25 GOMAXPROCS: 4
2017/10/27 11:20:25 Starting 4 threads
2017/10/27 11:20:35 Id routine: 1 rate: 106.382979
2017/10/27 11:20:35 Id routine: 2 rate: 102.040816
2017/10/27 11:20:35 Id routine: 0 rate: 125.000000
2017/10/27 11:20:35 Id routine: 3 rate: 117.647059
2017/10/27 11:20:35 Current total rate was 451.07 K/s
2017/10/27 11:20:35 Starting 4 threads
2017/10/27 11:20:45 Id routine: 1 rate: 111.111111
2017/10/27 11:20:45 Id routine: 2 rate: 112.359551
2017/10/27 11:20:45 Id routine: 0 rate: 119.047619
2017/10/27 11:20:45 Id routine: 3 rate: 128.205128
2017/10/27 11:20:45 Current total rate was 470.72 K/s
58.80user 2.20system 0:20.01elapsed 304%CPU
In summary, you appear to be benchmarking the Go garbage collector. Garbage collection is not free. Minimize your allocations. Use the Go testing package benchmark facilities.