Search code examples
wolfram-mathematicaparallel-processingevaluate

Evaluating cells in parallel in Mathematica


I have this many hundreds of cell long Mathematica file and I want to use parallel evaluation. I have a 2 processor x 4 core each machine with 16 Gb memory. My Mathematica license allows me to run at most 2 master kernels with 1 of the master could have 4 slave kernels (this is my interpretation after I played with it for a while).

I used to run my code in two master kernels in two different notebooks. To speed up things further, I tried to encapsulate a few cells with ParallelEvaluate[] and it seemed to work. Then I also have 4 copies of my code running unaware of each other through one of the master kernels, which is fine. (I am basically trying to run as many copies of my code/mathkernel in parallel as possible. I am not shooting for anything truly parallel yet).

Since my code is too long and complicated, I do not want to edit every cell again to make them evaluate in parallel. Is there anything magical I can put in the beginning of my notebook so every cell evaluated after that will be by default ParallelEvaluate[ ... cell contents.... ]?


Solution

  • As suggested by belisarius, $Pre=ParallelEvaluate does exactly what I want. One problem was when I do $Pre=Identity to turn it off so I can go back to my master kernel, mathematica still tries to evaluate that in slave kernels instead of master and fails. I ended up solving it as follows: SetSharedVariable[parallelcontrol]; parallelcontrol = ParallelEvaluate; $Pre := parallelcontrol; ... everything is evaluated in slaves here ... ; parallelcontrol = Identity; .... everything go back to be evaluated on master only ... Here is a sample run on my laptop which has 2 cores:

    LaunchKernels[]
    

    {KernelObject[1, "local"], KernelObject[2, "local"]}

    $KernelID
    

    0

    ParallelEvaluate[$KernelID]
    

    {1, 2}

    SetSharedVariable[parcontrol]; $Pre := parcontrol; parcontrol = ParallelEvaluate
    

    Null[ParallelEvaluate]

    $KernelID
    

    {1, 2}

    parcontrol = Identity
    

    {Identity, Identity}

    $KernelID
    

    0

    parcontrol = ParallelEvaluate
    

    ParallelEvaluate

    $KernelID
    

    {1, 2}