Search code examples
pythonrecommendation-enginecollaborative-filtering

When creating recommendation system for multiple types of products, how do you deal with bias in implicit rating?


When doing recommendation system (collaborative filtering) for retail business, there are no actual rating (like 1-10 satisfactory with the item).

Therefore, I use no. of times each user purchase a certain item as implicit rating. However, in doing so, there could create bias between slow-moving products (such as Television) and fast-moving products (such as Chips, Snacks) since the no. of times customers would buy the slow-moving products are much less than fast-moving ones.

My questions are :

  1. Is there anything better than no. of times a customer buys the certain items or visit counts to use as rating?
  2. How do you add weight to help lessen the bias between slow-moving and fast-moving items.

Thank you for your answer in advanced!


Solution

  • Attempting to answer your specific questions:

    1. Yes, but it depends on what other information you have about the items. I have tried to answer making some reasonable assumptions.

    2. I can presume that you have the cost of the item. You can normalise by the cost of the item to ensure some reduction (not complete removal) of bias. You can do a direct normalisation (weighted_rating = implicit_rating * cost per item). Here, the cost_per_item serve the purpose of weights to reduce bias. Alternatively, you can experiment with groups of prices by binning or clustering the prices across all items to form groups of products, thereby giving you a group_mean_price. This can then be used as weights.

    If you know something else about the item (such as perishable/ consumable/ bulky) you can have more representative clusters. Thereafter, you can assign implicit_rating to each item. Now, you can normalise each implicit_rating by using an aggregate statistics for the group rating (say group_mean_implicit_rating). So, weighted_rating = implicit_rating/group_mean_implicit_rating