I am a new user to try dask delayed. I want to use delayed to automatically transform function and code into Delayed. However, I found delayed.compute didn't recursively compute Delayed in collection...
from dask import delayed, base
@delayed
def inc(x):
return x + 1
@delayed
def colls(ind):
return [inc(i) for i in xrange(ind)]
data2 = colls(2)
data2.compute() # I expect [1, 2], but get [Delayed('inc-...'),
Delayed('inc-...')]
Did I missing any thing to make it work or Dask.delayed doesn't support it?
You are correct that you should not use delayed functions within other delayed functions (unless you are doing something very strange). However you can pass delayed values into other delayed functions.
In your particular example I would leave colls as not-delayed. You want it to immediately determine how many delayed inc calls to make. Generally you want to immediately call any code that builds out your task graph and delay any function that just does work.
from dask import delayed, compute
@delayed
def inc(x):
return x + 1
def colls(ind):
return [inc(i) for i in xrange(ind)]
data2 = colls(2)
compute(data2)
# [1, 2]