Spawning 1000 processes at the same time in Erlang

Question

I want to spawn 1000 or a variable number of processes in Erlang.

server.erl:

-module(server).
-export([start/2]).

start(LeadingZeroes, InputString) ->
    % io:format("Leading Zeroes: ~w", [LeadingZeroes]),
    % io:format("InputString: ~p", [InputString]).
    mineCoins(LeadingZeroes, InputString, 100).

mineCoins(LeadingZeroes, InputString, Target) ->
    PID = spawn(miner, findTargetHash(), []), % How to spawn this process 1000 times so that each process computes something and sends the results here
    PID ! {self(), {mine, LeadingZeroes, InputString, Target}},
    receive
        {found, Number} ->
            io:fwrite("Rectangle area: ~w", [Number]);
        % {square, Area} ->
        %     io:fwrite("Square area: ~w", [Area]);
        Other ->
            io:fwrite("In Other!")
    end.
    % io:fwrite("Yolo: ~w", [Square_Area]).

miner.erl (client):

-module(miner).
-export([findTargetHash/0]).

findTargetHash() ->
    receive
         {From , {mine, LeadingZeroes, InputString, Target}} ->
            % do something here
            From ! {found, Number};
        {From, {else, X}} ->
            io:fwrite("In Else area"),
            From ! {square, X*X}
    end,
    findTargetHash().

Here, I wish to spawn the processes, 1000 of them(miner), how does one achieve this? Through list comprehensions or recursion or any other way?

Answer 1

Generally, you can do something N times like this:

-module(a).
-compile(export_all).

go(0) ->
    io:format("!finished!~n");
go(N) ->
    io:format("Doing something: ~w~n", [N]),
    go(N-1).

In the shell:

3> c(a).   
a.erl:2:2: Warning: export_all flag enabled - all functions will be exported
%    2| -compile(export_all).
%     |  ^
{ok,a}

4> a:go(3).
Doing something: 3
Doing something: 2
Doing something: 1
!finished!
ok

If you need to start N processes and subsequently send messages to them, then you will need their pids to do that, so you will have to save their pids somewhere:

go(0, Pids) ->
    io:format("All workers have been started.~n"),
    Pids;
go(N, Pids) ->
    Pid = spawn(b, worker, [self()]),
    go(N-1, [Pid|Pids]).


-module(b).
-compile(export_all).

worker(From) ->
    receive
        {From, Data} ->
            io:format("Worker ~w received ~w.~n", [self(), Data]), 
            From ! {self(), Data * 3};
        Other ->
            io:format("Error, received ~w.~n", [Other])
    end.

To start N=3 worker processes, you would call go/2 like this:

Pids = a:go(3, []).

That's a little bit awkward for someone who didn't write the code: why do I have to pass an empty list? So, you could define a go/1 like this:

go(N) ->  go(N, []).

Then, you can start 3 worker processes by simply writing:

Pids = go(3).

Next, you need to send each of the worker processes a message containing the work they need to do:

do_work([Pid|Pids], [Data|Datum]) ->
    Pid ! {self(), Data},
    do_work(Pids, Datum);
do_work([], []) ->
    io:format("All workers have been sent their work.~n").

Finally, you need to gather the results from the workers:

gather_results([Worker|Workers], Results) ->
    receive
        {Worker, Result} ->
            gather_results(Workers, [Result|Results])
    end;
gather_results([], Results) ->
    Results.

A couple of things to note about gather_results/2:

1. The Worker variable in the receive has already been assigned a value in the head of the function, so the receive is not waiting for just any worker process to send a message, rather the receive is waiting for a particular worker process to send a message.

2. The first Worker process in the list of Workers may be the longest running process, and you may wait in the receive for, say, 10 minutes for that process to finish, but then getting the results from the other worker processes will require no waiting. Therefore, gathering all the results will essentially take as long as the longest process plus a few microseconds to loop through the other processes. Similarly, for other orderings of the longest and shortest processes in the list, it will only take a time equal to the longest process plus a few microseconds to receive all the results.

Here is a test run in the shell:

27> c(a).                                                       
a.erl:2:2: Warning: export_all flag enabled - all functions will be exported
%    2| -compile(export_all).
%     |  ^

{ok,a}

28> c(b).                                                       
b.erl:2:2: Warning: export_all flag enabled - all functions will be exported
%    2| -compile(export_all).
%     |  ^

{ok,b}

29> Pids = a:go(3, []).                                         
All workers have been started.
[<0.176.0>,<0.175.0>,<0.174.0>]

30> a:do_work(Pids, [1, 2, 3]).                                 
All workers have been sent their work.
Worker <0.176.0> received 1.
Worker <0.175.0> received 2.
Worker <0.174.0> received 3.
ok

31> a:gather_results(Pids, []).                                 
[9,6,3]