Search code examples
recommendation-enginegoogle-cloud-ml

Google Cloud ML for recommendations

I need to make a music recommendation system for a music platform using machine learning.

The platform has:

  • 40+ million songs
  • 4.5+ million albums
  • 2.5+ million artists
  • 1648 genres
  • 600k users

The relations between the above objects are:

  • 1-song - many-genres
  • 1-song - many-artists
  • 1-song - 1-album
  • 1-album - many-songs
  • 1-album - many-artists
  • 1-artist - many-albums
  • 1-artist - many-songs

And i got the users activity(listened songs) and favorites(song, artist, albums)

Amazon ML doesn't seem to support collaborative filtering, and now i'm looking through Google Cloud ML.

One problem is the size of the data. Basically every song has 1+ genres and 1+ artists which are categorical attributes. Amazon ML support ~100 categories(at a glance i have 2.5m if the artists are regarded as categories). Looking through google's machine learning pages i found only simple examples so i don't really know where to start.

Being a beginner in the machine learning landscape i wonder if the problem is the way i see(try to solve) these recommendations, or even if ML is the way to go.

Solution

  • Google CloudML Engine is a hosted solution for running TensorFlow programs. TensorFlow is a machine learning framework, designed with scale in mind. So as long as you can write a Distributed TensorFlow program, you can run it on CloudML Engine, which should allow you to scale quite well. (I will note, there is a learning curve both to TensorFlow and Machine Learning in general, but you'll definitely want an ML-based solution for recommendations).

    A quick Google search reveals multiple helpful materials for building a recommendation system using TensorFlow (caveat: I haven't vetted any of these):