Difference between distributed and non-distributed arrays in MATLAB

Question

Suppose that we have this code in MATLAB:

parpool('local',2) % Create a parallel pool
W = ones(6,6);
W = distributed(W); % Distribute to the workers
spmd
    T = W*2; % Calculation performed on workers, in parallel
    % T and W are both codistributed arrays here
end
T % View results in client.
whos % T and W are both distributed arrays here
delete(gcp) % Stop pool

1. I read in documentation that the difference between normal arrays and distributes array is : When we use distributed arrays, these arrays directly send to workers and there isn't any array on clients. So we don't have any access to these arrays in client? Is this only discrepancy?

2. What is the difference in structure and output of code if we remove W = distributed(W); line? What is purpose of using distributed array?

3. What is difference between distributed and codistributed. As i read in documentation we can only use codistributed in spmd block. Is that ture?

Answer 1

Distributed arrays are stored on the workers, not the client, and operations on them are carried out in parallel by the workers - that's the point of them.

The difference between distributed and codistributed arrays is only one of perspective. From the point of view of the client, they are distributed arrays; from the point of view of the workers, they are codistributed arrays.

To illustrate, first start a pool:

>> parpool('local',2)

create an array:

>> W = ones(6,6);

W is stored on the client.

Now create a distributed array from W:

>> V = distributed(W);

V is stored on the workers, split across each worker. You still have access to V from the client, but when you do so it is pulling V back from the workers.

>> V
V =
     1     1     1     1     1     1
     1     1     1     1     1     1
     1     1     1     1     1     1
     1     1     1     1     1     1
     1     1     1     1     1     1
     1     1     1     1     1     1 

Note that in the Workspace Browser, V is there as a 6x6 distributed array, not a 6x6 double like W.

Now although from the point of view of the client V is a distributed array, from the point of view of the workers, V is a codistributed array.

>> spmd; disp(V); end
Lab 1: 
          LocalPart: [6x3 double]
      Codistributor: [1x1 codistributor1d]

Lab 2: 
          LocalPart: [6x3 double]
      Codistributor: [1x1 codistributor1d]

You can see that V is codistributed, and that only half of it (6x3) is stored on each worker.

When you do something with V, that happens on the workers, in parallel, and the results are stored on the workers as a distributed/codistributed array:

>> spmd; T = V*2; end
>> spmd; disp(T); end
Lab 1: 
          LocalPart: [6x3 double]
      Codistributor: [1x1 codistributor1d]

Lab 2: 
          LocalPart: [6x3 double]
      Codistributor: [1x1 codistributor1d]

You have access to T from the client just as you did with V, but to explicitly bring it back, use gather:

>> S = gather(T);

Note that S is now a 6x6 double, not a distributed array.