Search code examples
multithreadingdictionaryf#parallel-processingblock

does multithread conflict with Map in F#


let len = 25000000
let map = Map.ofArray[|for i =1 to len do yield (i,i+1)|]
let maparr = [|map;map;map;map|]
let f1 i =   
    for i1 =1 to len do
        let l1 = maparr.[i-1].Item(i1)
        ()

let index = [|1..4|]
let _ = index |> Array.Parallel.map f1
printf "done" 

I found that only one core is working at full speed be the code above . But what i except is all the four thread is working together with a high level of cpu usage. So it seems multithread conflict with Map, am i right? If not, how can achieve my initial goal? Thank you in advance


Solution

  • So I think you were tripping a heuristic where the library assumed when there were only a small number of tasks, it would be fastest to just use a single thread.

    This code maxes out all threads on my computer:

    let len = 1000000
    let map = Map.ofArray[|for i =1 to len do yield (i,i+1)|]
    let maparr = [|map;map;map;map|]
    let f1 (m:Map<_,_>) =
        let mutable sum = 0
        for i1 =1 to len do
            let l1 = m.Item(i1)
            for i = 1 to 10000 do
                sum <- sum + 1
        printfn "%i" sum
    
    let index = [|1..40|]
    printfn "starting"
    index |> Array.map (fun t -> maparr.[(t-1)/10]) |> Array.Parallel.iter f1
    printf "done"
    

    Important changes:

    1. Reduced len significantly. In your code, almost all the time was spent creating the matrix.
    2. Actually do work in the loop. In your code, it is possible that the loop was optimised to a no-op.
    3. Run many more tasks. This tricked the scheduler into using more threads and all is good