How do I do I/O in Julia distributed for loop run non-interactively?

With the following code in foo.jl:

using Distributed

@distributed for k = 1:4
    println("Creating file ", k)
    write(string("file_", k), "foo")
end

executing include("foo.jl") in the REPL prints the expected lines and creates the expected files, but when I step out of the REPL and run

julia foo.jl

Nothing is written and no files are created.

Why is there a difference and what is required for the script to run as expected outside of the REPL?

Solution

As stated in the documentation:

Note that without a reducer function, @distributed executes asynchronously, i.e. it spawns independent tasks on all available workers and returns immediately without waiting for completion. To wait for completion, prefix the call with @sync

This is what happens, in your case (which does not use a reducer function): @distributed spawns tasks and returns immediately. Your tasks are so short that in the REPL you get to see them almost immediately, and don't really notice the difference with a synchronous process (except that perhaps the output is mixed with the REPL prompt).

In a script however, the julia main process terminates immediately after having spawned the tasks, without having any occasion to actually run them. And you don't get to see the output.

As advised by the documentation, use @sync to wait for the tasks to complete before existing:

using Distributed

@sync @distributed for k = 1:4
    println("Creating file ", k)
    write(string("file_", k), "foo")
end