- Types of Recommender Systems
- recommend things (products)
- recommemd content
- recommend music (such as pandora music genome project)
- recommend people (such as online dating system)
- recommend search results (personalize search result rather than just information retrieval)
- Some terminology
- top-N recommenders
- for example, content-based recommender has individual interests store -> candidate generation module -> <- item similarities store
- candidate generation -> candidate ranking module -> filtering module -> recommend results
- item similarities store is where all the magic happens
- candidate generation, candidate ranking, filtering will live in some distributed web-service. Web front-end will talk to the while rendering a page for specific user
- Recommendation system evaluation
- split data into training set. Train recommender system only using the training data and use test set to measure accuracy
- k-fold cross validation
- accuracy metric:
- mean absolute error (MAE):
where yi is the rating predicted by system and xi is the actual rating user gives. The low the better
- root mean square error (RMSE): penalize more when the predicted rating is too off, penalize less when the predicted rating is similar to the actual rating.
^2}{n}} "\sqrt{\frac{\sum _{i=1}^n (y_i - x_i)^2}{n}}")
. The low the better
- ideally we should measure how people react when giving a recommendation that people have never seen before. But this cannot be done offline
- some metric that achieve are: hit rate. hits/users
- leave-one-out cross validation
- average reciprocal hit rate (ARHR):
. It measures the system’s ability to recommend top items in users’ point of view.
- cumulative hit rate (cHR): throw away results if our predicted rating is below some threshold
- rating hit rate (rHR): split hit rate by different rating score
- converage metric
- percentage of <user, item> pairs that can be predicted
- diversity metric
- 1-S where S is average similarity between recommendation pairs
- novelty
- popularity rank of recommended items
- churn
- how often do recommemdataions change
- responsiveness
- how quickly does new user behavior influence your recommendations
- use online A/B test to see which metric is more important
- Content-based filtering
- consine similarity
- convert movie genres to dimensions
- k nearest neighbors
- get similarity scores between this movie and all others the user rated
- sort
- select top k nearest movies
- use similarity score to do the weighted average in order to get final rating prediction