Perl thread shared data

Question

In my script there are n worker threads (0,1..n-1) and each work on the Nth item of the following arrays. Input array is used to provide input to thread and output array accepts the output from the thread. A thread won't access other items of the array. In that case should I declare the array as shared ?

my @ThreadInput   :shared=();
my @ThreadOutput  :shared=();

Answer 1

(I shall name "caller" the thread that populates @ThreadInput and consumes @ThreadOutput.)

Perl variables aren't shared between threads unless marked with :shared. Each thread gets a copy of variables not marked with :shared.

So,

If the caller populates @ThreadInput before the workers starts, @ThreadInput does not need to be shared, but it will avoid creating a copy of the array for each worker if it is.

If the caller populates @ThreadInput after the workers starts, @ThreadInput must be shared. If it isn't, changes in the caller's @ThreadInput won't affect the worker's copy.

@ThreadOutput must be shared. If it isn't, changes in the worker's @ThreadOutput won't affect the caller's copy.

---

It's going to be very hard to reuse workers with that model. You should probably be using something more like the following:

use threads;
use Thread::Queue 1.03;   # or Thread::Queue::Any

use constant NUM_WORKERS => ...;

sub handle_request {
    my ($request) = @_;
    return ...response...;
}

{
   my $request_q  = Thread::Queue->new();
   my $response_q = Thread::Queue->new();

   my @threads;
   my $threads;
   for (1..NUM_WORKERS) {
      ++$threads;
      push @threads, async {
         while (my $request = $request_q->dequeue()) {
            $response_q->enqueue([ $request => handle_request($request) ]);
         }

         $response_q->enqueue(undef);
      };
   }

   ... Add stuff to queue $request_q->enqueue(...) ...

   $request_q->end();  # Can be done later if you want to add more items later.

   while ($threads && my $job = $response_q->dequeue()) {
      if (!defined($job)) {
         --$threads;
         next;
      }

      my ($request, $response) = @$job;
      ... handle response ...
   }

   $_->join for @threads;
}