Better Recommendation Engines

Researchers Devise Better Recommendation Algorithm 
Improved recommendation algorithm should work especially well when ratings data are “sparse.”

MIT News  By Larry Hardesty

Researchers at the Massachusetts Institute of Technology (MIT) have developed a new recommendation algorithm based on a theoretical analytic framework using cosine similarity, which they say should work better than current algorithms. The researchers note the algorithm should be especially effective when ratings data is "sparse." Sparse data means there may be so little overlap between users' ratings that cosine similarity is rendered meaningless, making it necessary to aggregate the data of many users. MIT professor Devavrat Shah says the framework assumes the relative weight a user assigns to ratings remains the same, and each user's function is running on the same set of features. Shah notes this yields sufficient consistency to extrapolate statistical inferences about the probability that one user's ratings will predict another's. The team used the framework to demonstrate that, in instances of sparse data, their "neighbor's-neighbor" algorithm should return more accurate predictions than any known algorithm.... "