Search code examples
windowsjuliastack-overflowjupyter-lab

How to increase stack size for Julia in Windows?


I wrote a recursive function (basically a flood fill), it works fine on smaller datasets, but for slightly larger input it throws StackOverflowError.

How to increase the stack size for Julia under Windows 10? Ideally the solution should be applicable to JupyterLab.

It's a singe use program, no point in optimizing/rewriting it, I just need to peak at the result and forget about the code.

Update: As a test case, I provide the following MWE. This is just a simple algorithm that recursively visits each cell of n by n array:

n = 120

visited = fill(false, (n,n))

function visit_single_neighbour(i,j,Δi,Δj)
  if 1 ≤ i + Δi ≤ n && 1 ≤ j + Δj ≤ n
    if !visited[i+Δi, j+Δj]
      visited[i+Δi, j+Δj] = true
      visit_four_neighbours(i+Δi, j+Δj)
    end
  end
end

function visit_four_neighbours(i,j)
  visit_single_neighbour(i,j,1,0)
  visit_single_neighbour(i,j,0,1)
  visit_single_neighbour(i,j,-1,0)
  visit_single_neighbour(i,j,0,-1)
end

@time visit_four_neighbours(1,1)

For n = 120 the output is 0.003341 seconds, but for n = 121 it throws StackOverflowError.

On a Linux machine with ulimit -s unlimited the code runs no problem for n = 2000 and takes about 2.4 seconds.

I've mirrored the question to Julia Discource: https://discourse.julialang.org/t/ow-to-increase-stack-size-for-julia-in-windows/79932


Solution

  • As you are no doubt aware Julia is not very optimized for recursion and the recommendation will probably always be to rewrite the code in some way.

    With that said there are of course ways to increase the stack limit. One undocumented way to achieve it from inside julia is to reserve stack space when creating a Task:

    Core.Task(f, reserved_stack::Int=0)
    

    Let's create a function wrapping such a task:

    with_stack(f, n) = fetch(schedule(Task(f,n)))
    

    for n = 2000 the following works on both windows and linux (as long as enough memory is available):

    julia> with_stack(2_000_000_000) do
      visit_four_neighbours(1,1)
    end