Search code examples
phpalgorithmvoting

Weighted voting system with karma

This question is more logic than programming at the moment.. once I understand what algorithm(s) I need to use I'll be looking into how to implement it.

I've got a list of items in a database that need to be voted up or down by users to determine if they are correct or not. The aim is to provide a % for each item to show how reliable the item information is.

There's a few criteria to take into account..

  • Votes are not absolute - each user’s vote weight depends on their karma.
  • User karma should be calculated based on their votes - for example, if the user submits an item and other users vote to confirm that it is correct, that user's karma would increase. Karma could also be given if a user votes for an item in the same direction that other users with high karma have voted. If they vote in the opposite direction to other users with high karma, their vote would be considered incorrect and although it'd lower the score of the item it'd also lower their karma level, making them less influential in future voting.
  • Users can cast negative votes as well as positive votes.
  • Calculated scores of items should take into account the age of the item (over time the score would decrease as the item could become less reliable).

Does anyone have any recommendations on the best algorithm(s) for doing this, or any tips on how to implement this in a programming language (such as PHP)?

Solution

  • Read this first: http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

    It's an introduction to a mathematical concept known as Wilson score confidence intervals for Bernoulli parameters.

    That article is a great primer on how to use your user's votes to calculate a score that is actually useful and mathematically sound. Do this, and you're already ahead of Amazon.com

    Then, I think you probably need to tweak that formula a bit. In that formula it uses p for the fraction of positive votes. You might need to update the formula for p, to reflect the karma of the user that cast that vote.

    Finally, to take the age into account, you multiply the outcome of the formula with an age multiplier. For example, if you want the outcome to become less relevant by 1% for each day it ages, multiply it by 0.99^age_in_days.

    In a nutshell, that is the path I would follow. Hope this helps.