Collaborative Metric Le Learning An Andy Hs Hsieh eh, Lo Longqi - - PowerPoint PPT Presentation

An Andy Hs Hsieh eh, Lo Longqi Yang, , Yi Yin Cui, Ts Tsung-Yi Yi Lin , Serge Be Belongie, D , Debo borah Es h Estri rin

Connected Experience Lab, Cornell Tech AOL CONNECTED EXPERIENCES LAB CORNELL TECH

Collaborative Metric Le Learning

1

Co Collaborative Me Metri ric Le Learn rning

• A different perspective on collaborative filtering
• Better accuracy
• Extremely efficient Top-K recommendations
• Easy to interpret and extend

2

Us User er-It Item em Matr trix ix

Users Items

3

Ma Matri rix Factori rization (MF MF)

≈

Users Items Users Items

4

Im Implic plicit it Feedbac eedback

• Ubiquitous in today’s online services
• Only positive feedback is available
• Traditional MF does not work

? ? ? ? ? ? ? ? ? ? ?

Click Thumbs up Like

5

Ma Matri rix Factori rization for r Imp mplicit Feedback

• Weighted Regularized Matrix Factorization (WR

WRMF MF) [Hu08]

• Probabilistic Matrix Factorization (PM

PMF) [Salakhutdinov08]

• Bayesian Personalized Ranking (BPR

BPR) [Rendle09] and many more …

6

Th Think Beyond Matrix

? ? ? ? ? ? ? ? ? ? ?

• No longer about estimating ratings
• But about modeling the relationships

between different user/item pairs Explicit Implicit

7

Th Think Beyond Matrix

• No longer about estimating ratings
• But about modeling the relationships

between different user/item pairs Explicit Implicit

8

Me Metri ric Le Learn rning

9

Known relationships Unknown relationships

Co Collaborative Me Metri ric Le Learn rning

• Learn a joint user-item distance metric.
• The Euclidean distances reflect the relationships between users/items.

10

Ba Based ed o

• n th

the i e inher eren ent T t Triangu gular In Ineq equality o ty of Metric c Learning – If If A is cl close to B, a , and nd B is cl close to to C, the , then n A is cl close to C.

• Fit the model with implicit feedback
• 1. An user is pulled closer to the items she liked
• 2. Other similar users are pulled closer.
• 3. The items users liked are also pulled closer.
• Top-K recommendations are simply KNN

search (a well-optimized task)

11

12

Ba Based ed o

• n th

the i e inher eren ent T t Triangu gular In Ineq equality o ty of Metric c Learning – If If A is cl close to B, a , and nd B is cl close to to C, the , then n A is cl close to C.

• Fit the model with implicit feedback
• 1. An user is pulled closer to the items she liked
• 2. Other similar users are pulled closer.
• 3. The items users liked are also pulled closer.
• Top-K recommendations are simply KNN

search (a well-optimized task)

13

Ba Based ed o

• n th

the i e inher eren ent T t Triangu gular In Ineq equality o ty of Metric c Learning – If If A is cl close to B, a , and nd B is cl close to to C, the , then n A is cl close to C.

• Fit the model with implicit feedback
• 1. An user is pulled closer to the items she liked
• 2. Other similar users are pulled closer.
• 3. The items users liked are also pulled closer.
• Top-K recommendations are simply KNN

search (a well-optimized task)

Co Collaborative La Large Ma Margin Nearest Neighbor r

User Positive item Imposter Safety Margin Gradients Before After

* The outline of figure is inspired by Weinberger, Kilian Q., John Blitzer, and Lawrence Saul. "Distance metric learning for large margin nearest neighbor classification." Advances in neural information processing systems 18 (2006): 1473. 14

Pitf Pitfalls alls of Matr trix ix Fac actoriz izatio tion (Dot-Pr Product)

• Dot-Product violates triangle inequality misleading embedding.

15

Pitf Pitfalls alls of Matr trix ix Fac actoriz izatio tion (Dot-Pr Product)

• Dot-Product violates triangle inequality misleading embedding.

𝑊

# $𝑊 % = 0: does not reflect that

they are both liked by 𝑉* 𝑉#

$𝑉% = 0: does not reflect that

they both share the same interest as 𝑉*

16

Co Collaborative Me Metri ric Le Learn rning Emb mbedding

• Euclidian distance faithfully reflects the relative relationships.

17

In Integ egrating ting It Item em Fea eatur tures es

• Use a learnable function (e.g.

Multi-Layer Perceptron) to project features into user-item embedding.

• Treat the projections as a prior

for items' locations.

18

Ev Evaluation

• 6 Datasets from Different Domains
• Pa

Papers - CiteULike

• Bo

Books - BookCrossing

• Ph

Photography - Flickr

• Ar

Arti ticles - Medium

• Mo

Movies - MovieLens

• Mu

Musi sic - EchoNest

19

Accuracy (Recall@5 @50)

• 40
• 20

20 40 60 80 100

CiteULike BookCX Flickr Medium MovieLens EchoNest Recall@50 Improvements Over BPR (%) WRMF WARP CML

* * * *

* Indicate that CML > the second best algorithm is statistically significant according to Wilcoxon signed rank test

20

Ac Accur uracy y (wi (with h Item Featur ures) s)

• 20

20 40 60 80 100 120

CiteULike BookCX Flickr Medium MovieLens VBPR CDL CML+F

* * *

* Indicate that CML > the second best algorithm is statistically significant according to Wilcoxon signed rank test

Recall@50 Improvements Over Factorization Machine (%)

21

Ef Efficiency

• All optimized with LSHs
• CML’s throughput is improved by 106x

with only 2% reduction in accuracy

• Over 8x faster than (optimized) MF

models given the same accuracy

8x faster

‘s are brute force search

22

Em Embe beddi dding ng Interpr pretabi bility

23

A B C A B C

Co Conclusi sions

• The notion of user-item matrix and matrix

factorization becomes less applicable with implicit feedback.

• CML is a metric learning model that has
• better accuracy, efficiency, interpretability,

and extensibility.

• Applying metric-based algorithms, such as

K-means, and SVMs, to other recommendation problems.

24

25

Thank you!

AOL CONNECTED EXPERIENCES LAB CORNELL TECH