pyspark ml recommendation - All recomendation

Question

Hy,

I'm new in Spark and I'm trying using ML recommendation.

My Code

df = sqlContext.createDataFrame(
[(0, 0, 4.0), (0, 1, 2.0), (1, 1, 3.0), (1, 2, 4.0), (2, 1, 1.0), (2, 2, 5.0)],
["user", "item", "rating"])

als = ALS(rank=10, maxIter=5)

model = als.fit(df)

model.userFactors.orderBy("id").collect()

How can I obtain 2 recommendation for all users for all movies?

thanks for you time.

Answer 1

It is not directly possible with ml.recommendation.ALSModel. You could use transform method

users = df.select("user").distinct()
items = df.select("item").distinct()

model.transform(users.join(items))

and filter the results afterwards but it is extremely inefficient. As far as I can tell it would be better to simply use mllib.recommendation.ALS here:

from pyspark.mllib.recommendation import ALS, Rating

model = ALS.train(df.rdd.map(lambda r: Rating(*r)), 10, 5)
model.recommendProductsForUsers(2)