Does Ray Offer any Functional/Declarative Interface to Map a Remote Function to an Iterator/Iterable?

Question

My present Code

#!/usr/bin/env python3
# encoding: utf-8
"""Demonstration of Ray parallelism"""
import ray
from typing import Iterator
ray.init()
@ray.remote
def square(n:int)->int:
    return n*n

references: Iterator[ray.ObjectRef] = map(lambda val: square.remote(val),
                                                    range(10))
ray.get([*references])
ray.shutdown()

Basically, nothing but a form of map(square, range(10)) powered by Ray.

Question

For such a standard and common pattern, the above operation looks too verbose. So does ray offer any API exposing a more declarative/functional interface to get the above result? In addition to map, best if it offers some kind of filter, reduce etc. as well.

Answer 1

Your example seems to have quite a lot of stuff which is not strictly necessary: type hints, the lambda function inside map, and the [*references] unpacking operator.

A minimal example with Ray is:

@ray.remote
def func(n):
    return n*n

refs = map(func.remote, range(10))
results = ray.get(refs)

Compare with the same minimal example without Ray:

def func(n):
    return n*n

results = map(func, range(10))

As you can see, the difference is just three things:

• the @ray.remote decorator
• passing func.remote instead of func to map()
• ray.get() to collect the results

I think this is simple enough that the Ray developers did not judge it to be necessary to have map() in the core API. However, there is ray.data.Dataset.map() which works on rows of a Dataset.