#!/usr/bin/env python3
# encoding: utf-8
"""Demonstration of Ray parallelism"""
import ray
from typing import Iterator
ray.init()
@ray.remote
def square(n:int)->int:
return n*n
references: Iterator[ray.ObjectRef] = map(lambda val: square.remote(val),
range(10))
ray.get([*references])
ray.shutdown()
Basically, nothing but a form of map(square, range(10))
powered by Ray.
For such a standard and common pattern, the above operation looks too verbose. So does ray offer any API exposing a more declarative/functional interface to get the above result? In addition to map, best if it offers some kind of filter, reduce etc. as well.
Your example seems to have quite a lot of stuff which is not strictly necessary: type hints, the lambda function inside map, and the [*references]
unpacking operator.
A minimal example with Ray is:
@ray.remote
def func(n):
return n*n
refs = map(func.remote, range(10))
results = ray.get(refs)
Compare with the same minimal example without Ray:
def func(n):
return n*n
results = map(func, range(10))
As you can see, the difference is just three things:
@ray.remote
decoratorfunc.remote
instead of func
to map()
ray.get()
to collect the resultsI think this is simple enough that the Ray developers did not judge it to be necessary to have map() in the core API. However, there is ray.data.Dataset.map() which works on rows of a Dataset.