I've started using ParallelDataTransfer.jl library (https://github.com/ChrisRackauckas/ParallelDataTransfer.jl) which is based on the solution given in Julia: How to copy data to another processor in Julia and I've come across a couple of questions which are listed as follows:
1- How to perform a computation on a worker using its own local variables? For example, let's define "x=pi" on process 2 using "sendto(2,x=pi)", now how to execute the function "sin(x)" on process 2 and put the result in variable "y" which also lives in process 2? (Note that when running "@spawnat 2 y=sin(x), the variable "y" doesn't appear in the scope of process 2 which can be accessed by running "@spawnat 2 whos()")
2- Following the first question, is it possible that a worker (say i) request a function to be executed on another worker (say j), using worker j's own local variables and then fetch the result on worker i? For instance in the above example, how can process 3 request process 2 to compute "sin(x)" where "x" lives in process 2, and then send the result to variable "z" which lives in process 3?
According to what Keegan Lensink suggested in a Julia Channel, these works can be done as follows:
1-
Define x on process 2
sendto(2, x = pi)
Evaluate the expression on 2, and store the result in Main
@fetchfrom 2 eval(Main, :(y = sin(x)))
Check that it worked by asking for y
@fetchfrom 2 y
2-
Process 3 is asking 2 to evaluate y = sin(x), then is assigning y to z in 3's module Main
@spawnat 3 eval(Main, :(z = eval(:(@fetchfrom 2 eval(Main, :(y = sin(x)))))))
@fetchfrom 3 z