Hybrid algorithms for recommending new items - - PowerPoint PPT Presentation
2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec 2011) Chicago, IL (USA) Oct 2011, 27 th Hybrid algorithms for recommending new items http://dx.doi.org/10.1145/2039320.2039325
Hybrid algorithms for recommending new items
2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec 2011)
Chicago, IL (USA) – Oct 2011, 27th
http://dx.doi.org/10.1145/2039320.2039325MOVIRI, R&D
ROBERTO TURRIN – Moviri, R&D
Paolo Cremonesi – Politecnico di Milano Fabio Airoldi – Moviri, R&D
http://dx.doi.org/10.1145/2039320.2039325 ..in a nutshell
requirements
experiments
Traditional recommender systems
Collaborative (CF) Content-based (CBF)
ProsCons
ProsCons
(since they do not have ratings)
(since user ratings are ignored)
..so CF or CBF? ..many variables
quality
CF CBF
time
new system mature system
CBF
?
TV domain: new items
many unrated, new TV programs
cannot be neglected
Existing hybrid algorithms
Several hybrid algorithms mix CF and CBF
(but also demographics, social)
e.g.:
. Melville, R. J. Mooney, and R. Nagarajan. “Content-boosted collaborative filtering for improved recommendations”, 2002
Filtering on the Web”, 2003
Pros
Some approaches show better quality than CF/CBFCons
Our hybrid algorithms
GOALS New-item Quality comparable to collaborative REQUIREMENTS: Batch/real-time scalability/complexity Updated recommendations Modularity: ability to re-use existing CF and CBF algorithms.Modularity: ability to re-use existing CF and CBF algorithms.
Implicit/explicit ratings Main contributions
GOALS New-item Quality comparable to collaborative REQUIREMENTS: Batch/real-time scalability/complexity Updated recommendations Modularity: ability to re-use existing CF and CBF algorithms.Two hybrid algorithms: extension of SimComb algorithm introduction of a new hybrid algorithm New-item stressing evaluation
Modularity: ability to re-use existing CF and CBF algorithms.
Implicit/explicit ratings MOVIRI, R&D
STATE-OF-THE-ART RECOMMENDER ALGORITHMS
Collaborative algorithms
Implemented strategies:
Item-item neighborhood-based (NNCos)
User Rating Matrix (URM)
Rating given by user u to item i In implicit dataset is either 1 or 0 u i
Item-item neighborhood-based (NNCos)
cosine metric
Latent factor models (PureSVD)
means of a matrix factorization (SVD)
Content-based algorithm
Weight of feature f in item i.
Computed as TF-IDF Example of features: genre, actors,directors,…
Item-content matrix (ICM)
f i
LSA (Latent Semantic Analysis) The ICM is factorized by means of SVD in order to discover latent semantic
Hybrid algorithms
Interleaved (INTL)
formed by alternating items recommended by the CF algorithm with items recommended by the CBF algorithm Item A Item B Item C Item Z Item Y Item X
CF list CBF list
Item A Item Z Item B Item Y
SimComb [Mobasher et al. 2004]
CF item-item similarities
Item Y
(1-α) + α
CBF item-item similarities
=
Hybrid list
HYBRID item-item similarities
MOVIRI, R&D
PROPOSED HYBRID ALGORITHMS
Collaborative filtering as main brick
CF
We trust CF recommendations when the model has been trained with “enough” information (i.e., ratings)
CBF
We add CBF-based data (i.e., rating) for better training the CF when no enough information is available
Collaborative filtering as main brick
CF
We trust CF recommendations when the model has been trained with “enough” information (i.e., ratings)
CBF
We add CBF-based data (i.e., features) for better training the CF when no enough information is available
Item-item model
KNN Item-item similarity matrix
i
j
A number of recommendation (CF and CBF) algorithms allow to compute item-item similarity.
Item-item model: real-time recommendations
KNN Item-item similarity matrix
i
+
User ratings
? ? ? ?
j
Item-item model: real-time recommendations
+
User ratings
? ? ? ?
Real-time requirements:
#items
MODEL
Filtered Feature Augmentation (FFA)
Motivation
Idea: add pseudo-ratings to the item profiles
CONTENT
CBF
Filter
RATINGS
CF
Model
Filtered Feature Augmentation (FFA)
Motivation
Idea: add pseudo-ratings to the item profiles
Entropy-based filtering (e.g., Gini impurity measure) predicted ratings
CONTENT
CBF
Filter
RATINGS
CF
Model
Similarity Injection Knn (SIMinjKnn)
Motivation
Idea: mixing CF and CBF similarities
CONTENT
CBF
CBF
CONTENT
CBF
RATINGS
CF
CF Model Combiner Model Model
MOVIRI, R&D
EVALUATION
Datasets
1M Movielens ~6K users, ~3.9K items, 1M ratingsML
An implicit, binary dataset collected from15’000 IPTV users over a period of six months
~15K users, ~800 rated items/~4K, ~26K ratings Multilanguage (mainly German, French) contentdata
TV
available at http://home.dei.polimi.it/cremones/memo/downloads/TV2.zip Testing methodology (1)
Training set (extracted from H1) Test set
extracted from H1
Discarded ratings
Testing methodology (1)
Training set (extracted from H1) Test set
extracted from H1
Discarded ratings
Testing methodology (2)
For each <user, item> <u,i> in H1+2:
Generate rating prediction for i Generate rating prediction for every other items Sort the items according to predicted ratingThere is a “hit” if rank(i) < N
There is a “hit” if rank(i) < N
i.e., item i appears in the top-N.In our tests, N=20
Non-hybrid algorithms
ML TV
ML TV
Hybrid algorithms: ML
ML
Hybrid algorithms: ML
ML
Hybrid algorithms: TV
TV
Toy sample
IJ2 BF1 BF2 IJ2 BF1 BF2
IJ1 BF3
CBF
IJ1 BF3
CF
Toy sample
IJ2 BF1 BF2
SIMInj
IJ2 BF1 BF2
FFA
CBF CF new “connections”
IJ1 BF3 IJ1 BF3
Conclusions / Future work
Proposed 2 hybrid algorithms: Higher recall than CF and CBF in the presence of newitems
Scalable / non-affecting real-time performance Handling implicit/explicit ratings Future work: Subjective evaluation Improving the filter with other information Other domains Roberto TURRIN, PhD
Moviri, R&D – Italy
roberto.turrin@moviri.com
Thank you
Politecnico di Milano
www.contentwise.tvroberto.turrin@moviri.com