A Kernel-Based Approach to Exploiting Interaction-Networks in - - PowerPoint PPT Presentation

▶
a kernel based approach to exploiting interaction
SMART_READER_LITE
LIVE PREVIEW

A Kernel-Based Approach to Exploiting Interaction-Networks in - - PowerPoint PPT Presentation

Oct 18, 2023 137 likes •579 views

A Kernel-Based Approach to Exploiting Interaction-Networks in Heterogeneous Information Sources for Improved Recommender Systems Oluwasanmi (Sanmi) Koyejo Joydeep Ghosh ECE Dept., University of Texas at Austin HetRec 11, October 27, 2011,

slide-1
SLIDE 1

A Kernel-Based Approach to Exploiting Interaction-Networks in Heterogeneous Information Sources for Improved Recommender Systems

Oluwasanmi (Sanmi) Koyejo Joydeep Ghosh

ECE Dept., University of Texas at Austin

HetRec ’11, October 27, 2011, Chicago, IL, USA

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 1 / 20

slide-2
SLIDE 2

Outline

1

Motivation

2

Modeling Approach

3

Experiments

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 2 / 20

slide-3
SLIDE 3

Outline

1

Motivation

2

Modeling Approach

3

Experiments

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 3 / 20

slide-4
SLIDE 4

Interaction Networks

courtesy flickr/yankeeincanada Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 3 / 20

slide-5
SLIDE 5

Interaction Networks and Recommender Systems

*Proposed approaches include [Golbeck, 2005, Jamali and Ester, 2009, Ma et al., 2008, Jamali and Ester, 2010]

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 4 / 20

slide-6
SLIDE 6

Many New Interactions

Social network(s) User implicit feedback Item category Item history

blog.spoongraphics.co.uk Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 5 / 20

slide-7
SLIDE 7

Need a New Approach

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 6 / 20

slide-8
SLIDE 8

Need a New Approach

Is the data always helpful?

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 6 / 20

slide-9
SLIDE 9

Need a New Approach

Is the data always helpful? data may be noisy

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 6 / 20

slide-10
SLIDE 10

Need a New Approach

Is the data always helpful? data may be noisy data may be corrupted or inaccurate

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 6 / 20

slide-11
SLIDE 11

Need a New Approach

Is the data always helpful? data may be noisy data may be corrupted or inaccurate data may be descriptive, but uncorrelated with target task

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 6 / 20

slide-12
SLIDE 12

How To Select the Useful Information?

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 7 / 20

slide-13
SLIDE 13

How To Select the Useful Information?

Use everything! noisy or uncorrelated data may confuse model

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 7 / 20

slide-14
SLIDE 14

How To Select the Useful Information?

Use everything! noisy or uncorrelated data may confuse model Use domain knowledge might miss hidden correlations

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 7 / 20

slide-15
SLIDE 15

How To Select the Useful Information?

Use everything! noisy or uncorrelated data may confuse model Use domain knowledge might miss hidden correlations

Goal

A model based approach for extracting useful information from multiple interaction networks.

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 7 / 20

slide-16
SLIDE 16

Outline

1

Motivation

2

Modeling Approach

3

Experiments

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 8 / 20

slide-17
SLIDE 17

Linear Prediction Model

zi,j is observed preference of the ith user for the jth item User feature: xi Item feature: yj Linear prediction model: ˆ zi,j = w⊤(xi ⊗yj) = x⊤

i Wyj.

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 8 / 20

slide-18
SLIDE 18

Generalized Matrix Factorization

Enforce that W have some maximum rank R W = UV⊤

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 9 / 20

slide-19
SLIDE 19

Generalized Matrix Factorization

Enforce that W have some maximum rank R W = UV⊤ Resulting prediction: ˆ zi,j = x⊤

i UV⊤yj

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 9 / 20

slide-20
SLIDE 20

Generalized Matrix Factorization

Enforce that W have some maximum rank R W = UV⊤ Resulting prediction: ˆ zi,j = x⊤

i UV⊤yj

Aside: if features are user and item indices ei,ej: ˆ zi,j = e⊤

i UV⊤ej = ui ⊤vj

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 9 / 20

slide-21
SLIDE 21

Extract Interaction Graph Features

Graph G = (V ,E ,A) V = {vi} represents the set of entities as vertices E = {ei,j} represents the set of links A represents strength of links. Assume ai,j > 0, ai,j = aj,i.

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 10 / 20

slide-22
SLIDE 22

Extract Interaction Graph Features

Graph G = (V ,E ,A) V = {vi} represents the set of entities as vertices E = {ei,j} represents the set of links A represents strength of links. Assume ai,j > 0, ai,j = aj,i.

Smooth Functions on the graph f : V → R

f is smooth on G if the average difference (f (vi)−f (vj))2 is small for close points vi and vj [Belkin et al., 2006].

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 10 / 20

slide-23
SLIDE 23

Normalized Laplacian

L =D− 1

2 (D−A)D− 1 2

=I−D− 1

2 AD− 1 2 D is diagonal di,i = j ai,j Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 11 / 20

slide-24
SLIDE 24

Normalized Laplacian

L =D− 1

2 (D−A)D− 1 2

=I−D− 1

2 AD− 1 2 D is diagonal di,i = j ai,j

Eigenvectors of L are smooth on G [Smola and Kondor, 2003]

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 11 / 20

slide-25
SLIDE 25

Normalized Laplacian

L =D− 1

2 (D−A)D− 1 2

=I−D− 1

2 AD− 1 2 D is diagonal di,i = j ai,j

Eigenvectors of L are smooth on G [Smola and Kondor, 2003] Eigen-decompose L =

N

  • j=1

λjηjηj

⊤

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 11 / 20

slide-26
SLIDE 26

Augmented Features

Compute first Dk eigenvectors of L from each Gk

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 12 / 20

slide-27
SLIDE 27

Augmented Features

Compute first Dk eigenvectors of L from each Gk Append features: xi =       ei η1(i) . . . ηK (i)      

  • Identity
  • Eigenvector G1

. . .

  • Eigenvector GK

Same for item side to compute yj Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 12 / 20

slide-28
SLIDE 28

Augmented Features

Compute first Dk eigenvectors of L from each Gk Append features: xi =       ei η1(i) . . . ηK (i)      

  • Identity
  • Eigenvector G1

. . .

  • Eigenvector GK

Same for item side to compute yj

Expensive! dimension grows with number of basis vectors Dk and number

  • f graphs K.

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 12 / 20

slide-29
SLIDE 29

Linear Kernels

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 13 / 20

slide-30
SLIDE 30

Linear Kernels

Kp(i,j) = xi,pxj,p and Gq(i,j) = yi,qyj,q

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 13 / 20

slide-31
SLIDE 31

Linear Kernels

Kp(i,j) = xi,pxj,p and Gq(i,j) = yi,qyj,q Separate kernel for each dimension xp and yq Let K0 = I and G0 = I

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 13 / 20

slide-32
SLIDE 32

Linear Kernels

Kp(i,j) = xi,pxj,p and Gq(i,j) = yi,qyj,q Separate kernel for each dimension xp and yq Let K0 = I and G0 = I

Kernel uses R×Nx user factor parameters vs. R× k Dk for linear model Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 13 / 20

slide-33
SLIDE 33

Linear Kernels

Kp(i,j) = xi,pxj,p and Gq(i,j) = yi,qyj,q Separate kernel for each dimension xp and yq Let K0 = I and G0 = I

Kernel uses R×Nx user factor parameters vs. R× k Dk for linear model

Combine: K =

P

  • p=1

apKp and G =

Q

  • q=1

aqGq {ap ≥ 0,

  • p

ap = 1} , {bq ≥ 0,

  • q

bq = 1}

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 13 / 20

slide-34
SLIDE 34

Outline

1

Motivation

2

Modeling Approach

3

Experiments

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 14 / 20

slide-35
SLIDE 35

Lastfm

Data

User-Artist Listen counts Nx = 1982,Ny = 17632,N = 92834 Interaction: P = 151,Q = 101:

User Social network D = 50 User-Tag-Artist graph; converted to user-user D = 100 and item-item D = 100

2 4 6 8 10 12 14

Log(Listen count)

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

Probability

Figure: Histogram of log transformed listen counts in Last.fm µ = 5.469, σ = 1.531.

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 14 / 20

slide-36
SLIDE 36

Lastfm

Results

Rank=5 Rank=10 Global Model 1.502 (0.014)

  • Uniform Interaction

1.492 (0.015) 1.498 (0.008) MF 1.139 (0.006) 1.173 (0.010) Weighted Interaction 1.071 (0.009) 1.106 (0.006)

Table: Average (std.) cross validation RMSE on Last.fm

Uniform: ap =

1 P+1,bq = 1 Q+1

MF: a0 = 1,b0 = 1

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 15 / 20

slide-37
SLIDE 37

Factor Sparsity

20 40 60 80 100 120 140

User eigen-kernel index

20 15 10 5

Log(weight)

Figure: Last.fm user kernel log(weight). Identity(1%), User social network (93%), Tags (6%)

20 40 60 80 100

Item eigen-kernel index

25 20 15 10 5

Log(Weight)

Figure: Last.fm item kernel log(weight). Identity(77%), Tags (23%)

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 16 / 20

slide-38
SLIDE 38

Movielens

Data

User movie ratings [0.5,1,...5] Nx = 2113,Ny = 10197,N = 855598 Interaction: P = 101,Q = 321:

User-Tag-Artist graph; converted to user-user D = 100 and item-item D = 100 Movie-Actor D = 100, Movie-Director D = 100, Movie Genre D = 20

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 17 / 20

slide-39
SLIDE 39

Movielens

Results

Rank=5 Rank=10 Global Model .9478 (0.0018)

  • Uniform Interaction

.9471 (0.018) .9481 (0.0025) MF .7749 (0.0023) .7706 (0.0016) Weighted Interaction .7742 (0.0024) .7691 (0.001)

Table: Average (std.) cross validation RMSE on Movielens

Rank=5 Global Model .9185 Uniform Interaction .9184 MF .8790 Weighted Interaction .8728

Table: Global time split test RMSE on Movielens

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 18 / 20

slide-40
SLIDE 40

Factor Sparsity

20 40 60 80 100

User eigen-kernel index

25 20 15 10 5

Log(weight)

Figure: Movielens user kernel log(weight). Identity(96%), Tags (6%).

50 100 150 200 250 300

Item eigen-kernel index

25 20 15 10 5

Log(Weight)

Figure: Movielens item kernel log(weight). Identity(98%), Tags (.8%), Actor (.3%), Director (.3%), Genre (.08%).

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 19 / 20

slide-41
SLIDE 41

Conclusion

Model based approach for extracting information from pair-wise interaction networks. Results indicate which graphs may be most correlated with recommendation. Model is robust and degenerates to the matrix factorization model when no relevant interactions are found.

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 20 / 20

slide-42
SLIDE 42

Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399–2434, 2006. Jennifer Ann Golbeck. Computing and applying trust in web-based social

  • networks. PhD thesis, College Park, MD, USA, 2005. AAI3178583.

Mohsen Jamali and Martin Ester. Trustwalker: a random walk model for combining trust-based and item-based recommendation. In Proceedings

  • f the 15th ACM SIGKDD international conference on Knowledge

discovery and data mining, KDD ’09, pages 397–406, New York, NY, USA,

  • 2009. ACM. ISBN 978-1-60558-495-9. doi:

http://doi.acm.org/10.1145/1557019.1557067. URL

❤tt♣✿✴✴❞♦✐✳❛❝♠✳♦r❣✴✶✵✳✶✶✹✺✴✶✺✺✼✵✶✾✳✶✺✺✼✵✻✼.

Mohsen Jamali and Martin Ester. A matrix factorization technique with trust propagation for recommendation in social networks. In Proceedings

  • f the fourth ACM conference on Recommender systems, RecSys ’10, pages

135–142, New York, NY, USA, 2010. ACM. ISBN 978-1-60558-906-0. Hao Ma, Haixuan Yang, Michael R. Lyu, and Irwin King. Sorec: social recommendation using probabilistic matrix factorization. In Proceeding

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 20 / 20

slide-43
SLIDE 43
  • f the 17th ACM conference on Information and knowledge management,

CIKM ’08, pages 931–940, New York, NY, USA, 2008. ACM. ISBN 978-1-59593-991-3. doi: http://doi.acm.org/10.1145/1458082.1458205. URL ❤tt♣✿✴✴❞♦✐✳❛❝♠✳♦r❣✴✶✵✳✶✶✹✺✴✶✹✺✽✵✽✷✳✶✹✺✽✷✵✺.

  • A. J. Smola and I.R. Kondor. Kernels and regularization on graphs. In
  • B. Schölkopf and M. Warmuth, editors, Proceedings of the Annual

Conference on Computational Learning Theory, Lecture Notes in Computer Science. Springer, 2003. Manolopoulos Y. Symeonidis P ., Tiakas E. Product recommendation and rating prediction based on multi-modal social networks. In Proceedings

  • f the 5th ACM Conference in Recommender Systems (RecSys’2011),

Chicago, IL, 2011. URL

❤tt♣✿✴✴❞❡❧❛❜✳❝s❞✳❛✉t❤✳❣r✴♣❛♣❡rs✴r❡❝s②s✵✹✹✲s②♠❡♦♥✐❞✐s✳♣❞❢.

Koyejo & Ghosh (UT Austin) Kernels for Interaction Networks HetRec ’11 20 / 20