I'm trying to tile an image and apply calculations to each tile in parallel. But it's not behaving like I expect. I made it print 'here' whenever the function is executed, and many are printed quickly, which indicates they're being launched simultaneously. But my cpu load in my task manager never gets above 100%, and it takes a long time to execute. Can someone please advise? This is my first time using skimage.util.apply_parallel, which uses Dask.
from numpy import random, ones, zeros_like
from skimage.util import apply_parallel
def f3(im):
print('here')
for _ in range(10000):
u=random.random(100000)
return zeros_like(im)
if __name__=='__main__':
im=ones((2,4))
f = lambda img: f3(img)
im2=apply_parallel(f,im,chunks=1)
I looked at the source code, and apply_parallel relies on this Dask command:
res = darr.map_overlap(wrapped_func, depth, boundary=mode, dtype=dtype)
But I found that it needs .compute('processes')
at the end of it to guarantee multiple cpu's. So now I'm just using Dask itself:
import dask.array as da
im2 = da.from_array(im,chunks=2)
proc = im2.map_overlap(f, depth=0).compute(scheduler='processes')
Then the cpu usage really jumps!