Data Context Adaptation for Accurate Recommendation with Additional - - PowerPoint PPT Presentation

Hyunsik Jeon (SNU) 1

Data Context Adaptation

for Accurate Recommendation with Additional Information

Hyunsik Jeon, Bonhun Koo, and U Kang Seoul National University IEEE BigData 2019

Hyunsik Jeon (SNU) 2

Outline

n Introduction n Proposed Method n Experiments n Conclusion

Hyunsik Jeon (SNU) 3

Recommendation Systems

… …

5 1 5 1 5

friendship

3

friendship Ratings

5 Fantasy Action Drama

Hyunsik Jeon (SNU) 4

Recommendation Systems

… …

5 1 5 1 5

friendship

3

friendship Ratings

5 Fantasy Action Drama Recommendation

Hyunsik Jeon (SNU) 5

Problem Definition

(Data Context-Aware Recommendation)

n Given: rating matrix !, an auxiliary matrix "

q !: sparse rating matrix q ": social networks or item-genre relationships

n Goal: to predict unseen rating values in !

q Users want to be provided items that they will give

high ratings.

5 ？ ？ 2 1 ？ ？ 2 ？ ？ 3 ？ ？ ？ ？ ？ 3 1 ？ ？ 5 ？ ？ 2 ？

User-Movie Matrix User Item 1 2 3 4 5 1 2 3 4 5

1 1 1 1 1 1 1 1 1

Movie-Genre Matrix

1 1 1 1 1 1 1 1 1 1

User-User Matrix Item Genre 1 2 3 4 5 1 2 3 4 5 User User 1 2 3 4 5 1 2 3 4 5

! matrix " matrices

Hyunsik Jeon (SNU) 6

Collective Matrix Factorization

n Collective Matrix Factorization (CMF) is the

most dominant method in data context-aware recommendation

n Key idea of CMF

q Factorize two matrices while sharing the common

latent factor

User-Item Rating Matrix ! user " item # !$,& ≈

($

Item-Genre Matrix ) ≈ )

&,*

item # genre +

,

&

• inner-product

.*

• inner-product

,

&

user item item genre

Hyunsik Jeon (SNU) 7

Collective Matrix Factorization

n Inference and loss

q !

"#$ = &#

'( $, !

+

$, = ( $ '-,

q . =

/ 0 ∑ #,$ ∈34 !

"#$ − "#$

0 + / 0 ∑ $,, ∈37 !

+

$, − + $, 0 + 8 0 ( & : 0 + ( : 0 + - : 0), where

n " is rating matrix, + is additional matrix, n & is user latent matrix, ( is item latent matrix, n - is additional context matrix (e.g., genre).

Details

User-Item Rating Matrix ! user " item # !$,& ≈

($

Item-Genre Matrix ) ≈ )

&,*

item # genre +

,

&

• inner-product

.*

• inner-product

,

&

user item item genre

Hyunsik Jeon (SNU) 8

Collective Matrix Factorization

n CMF is extended to biased-CMF if bias terms

are added.

q !

"#$ = &#

'( $ + *+ + *,, !

/

$0 = ( $ '10 + 2

*, + 2 *3

q 4 =

5 6 ∑ #,$ ∈9: !

"#$ − "#$

6 + 5 6 ∑ $,0 ∈9< !

/

$0 − / $0 6 + = 6 ( & ? 6 + ( ? 6 + 1 ? 6)

q where

n " is rating matrix, / is additional matrix, n & is user latent matrix, ( is item latent matrix, n 1 is additional context matrix (e.g., genre), n *+, *,, 2

*,, and 2 *3 are 1-dimensional bias terms.

Details

Hyunsik Jeon (SNU) 9

Motivation

n Previous works have the following limitations:

q 1) Lack of consideration for the fact that data

contexts of rating auxiliary matrices are different

q 2) Restricted capability of expressing independent

information of users or items (e.g., biases)

q 3) To predict entries via an inner-product (linear)

How to address these limitations?

Hyunsik Jeon (SNU) 10

Outline

n Introduction n Proposed Method n Experiments n Conclusion

Hyunsik Jeon (SNU) 11

Key Ideas

n A novel approach for data context-aware

recommendation

n 1) Data context adaptation by !

" and ! #

q To consider differences between $ and %

n 2) Latent interaction/independence factors

q No size limit for latent independence factors

n 3) Fully-connected neural networks &

" and & #

q To model non-linear relationships

Hyunsik Jeon (SNU) 12

Overall Architecture

n Factorize data matrices ! and "

Item-Genre Data Context Item-Genre Matrix ! ≈ !

#,%

item & genre ' Rating Data Context () … * +,,# (-.

# ∘ (-0%‖. #

• ‖0%
• User

Item Genre ∘ Element-wise product ‖ Concatenation Rating Matrix * *,,# ≈ user 2 item & () (- (- Data Context Adaptation … MLP layer MLP layer (-.

# ∘ (-0%

3

, ) . # ) () 3 , ∘ (). #

()3

, ∘ (). #

().

#

()3

,

.

# )

4,

)

3

,

.

#

(-.

#

0% (-0% .

#

• 5%
• !

+

#,%

Hyunsik Jeon (SNU) 13

Item-Genre Data Context Item-Genre Matrix ! ≈ !

#,%

item & genre ' Rating Data Context () … * +,,# (-.

# ∘ (-0%‖. #

• ‖0%
• User

Item Genre ∘ Element-wise product ‖ Concatenation Rating Matrix * *,,# ≈ user 2 item & () (- (- Data Context Adaptation … MLP layer MLP layer (-.

# ∘ (-0%

3

, ) . # ) () 3 , ∘ (). #

()3

, ∘ (). #

().

#

()3

,

.

# )

4,

)

3

,

.

#

(-.

#

0% (-0% .

#

• 5%
• !

+

#,%

Latent Factors

n Latent interaction/independent factors

q Participate in different ways to predict an entry

latent independence vector latent interaction vector

Hyunsik Jeon (SNU) 14

Data Context Adaptation

n Any models can be used as adaptation

functions !

" and ! #

q Our choice is a linear projection:

n !

" $% = '($%, ! " ) *

= '()

*, !# ) *

= '+)

*, and !# ,- =

'+,-

q where ." is an projection matrix for / q .# is an projection matrix for 0 q $%, ) *, and ,- are latent interaction vectors

Hyunsik Jeon (SNU) 15

Non-linear Modeling

n Any non-linear models can be used as

predictive functions !

" and ! #

q Our choice is a multilayer perceptron (MLP):

n

$ %&' = !

"(

*"+& ∘ *"-

'

+&

"

• '

"

), $ /

'0 = ! #(

*#-

' ∘ *#10

• '

#

10

#

)

q where bracket 2 denotes concatenation of vectors q Tanh as activation functions in ! " and ! # q outputs of ! " and ! # are predicted ratings (scalars)

Hyunsik Jeon (SNU) 16

Loss Function

n Minimize two reconstruction errors for ! and "

together

q # = 1 − ' ()**+ + '()**-

n ()**+ =

. / ∑ 1,3 ∈56 7

!13 − !13

/ + 8 / 9:;+

n ()**- =

. / ∑ 3,< ∈5= 7

"

3< − " 3< / + 8 / 9:;-

q where Ω+ and Ω- are observable entries in ! and ", resp. q 9:;+ and 9:;- are #2-regularizations for ! and ", resp. q ' controls the balance of gradients from ()**+ and ()**-

Hyunsik Jeon (SNU) 17

Regularization

n Regularization terms

q !"#$ = ∑'∈)

*'

$ + + -$*' + +

∑.∈ℐ( 1

. $ + + -$1 . +) + ∑ 345 +

q !"#6 = ∑.∈ℐ

1

. 6 + + -61 . + +

∑7∈8( 97

6 + + -697 +) + ∑ 34: +

n

; + is Frobenius norm of vectors and matrices

n ) is set of users n ℐ is set of items n 8 is set of genres

Hyunsik Jeon (SNU) 18

Outline

n Introduction n Preliminaries n Experiments n Conclusion

Hyunsik Jeon (SNU) 19

Experiments

n Experimental questions

q Q1. Overall performance

n How better is our method compared to competitors?

q Q2. Effects of data context adaptation

n How does data context adaptation layer affect the

performance?

q Q3. Effects of interaction/independence factors

n How do dimensions of interaction/independence vectors

affect the performance?

q Q4. Neural Networks

n Do deeper structures yield better performance? n Does the activation function help improve performance?

Hyunsik Jeon (SNU) 20

Datasets

n 3 user-coupled datasets

q social network for additional data

n 3 item-coupled datasets

q item-genre relationships for additional data

Hyunsik Jeon (SNU) 21

Competitors

n Comparison of our method and competitors

Hyunsik Jeon (SNU) 22

Evaluation Metrics

n RMSE (Root Mean Square Error)

∑" ∑# $ %"# − %"#

'

()*( +,(-./*

n MAE (Mean Absolute Error)

∑" ∑# | $ %"# − %"#| ()*( +,(-./*

Hyunsik Jeon (SNU) 23

Experimental Results

n Q1. Overall performance

q Our method provides the best accuracy

Hyunsik Jeon (SNU) 24

Experimental Results

n Q2. Effects of data context adaptation

q !"#ℎ%&#'(: no adaptation to each context q )*+: separate adaptation for each entity

Hyunsik Jeon (SNU) 25

Experimental Results

n Q3. Effects of interaction/independence factors

q The total capacity of model is fixed

Hyunsik Jeon (SNU) 26

Experimental Results

n Q4-1. Neural Networks (deepness)

q !: DaConA with depth !

Hyunsik Jeon (SNU) 27

Experimental Results

n Q4-2. Neural Networks (activation functions)

q !"#ℎ%&#'(: DaConA without activation functions

Hyunsik Jeon (SNU) 28

Extension

n Using multiple auxiliary information

q Ratings, social information, and genre information 0.9 0.92 0.94 0.96 0.98 1 1.02 1.04 1.06 1.08 40 50 60 70 80 90 RMSE training set (%) DaConA Hybrid-CDL CMF Biased-CMF

• 9.4%

Best

• 11.1%
• 6.3%

Hyunsik Jeon (SNU) 29

Outline

n Introduction n Proposed Method n Experiments n Conclusion

Hyunsik Jeon (SNU) 30

Conclusion

n We propose a novel approach for data context-

aware recommendation

q Additional information is given as well as ratings

n Our key ideas:

q 1) Data context adaptation q 2) Latent interaction/independence factors q 3) Non-linear modeling

n DaConA outperforms the SOTA algorithms n Extensive experiments show our ideas help

improve performance

Hyunsik Jeon (SNU) 31

Thank you !

https://datalab.snu.ac.kr/dacona