CS490W: What is Collaborative Filtering? Collaborative Filtering - - PDF document

▶
cs490w what is collaborative filtering
SMART_READER_LITE
LIVE PREVIEW

CS490W: What is Collaborative Filtering? Collaborative Filtering - - PDF document

Mar 21, 2024 552 likes •628 views

CS490W: What is Collaborative Filtering? Collaborative Filtering (CF): Making recommendation decisions for a specific CS-490W user based on the judgments of users with similar tastes Collaborative Filtering Luo Si Train_User 1 1 5 3 3 4

slide-1
SLIDE 1

CS-490W

Collaborative Filtering

Luo Si Department of Computer Science Purdue University

CS490W: Abstract

Outline

Introduction to collaborative filtering Main framework Memory-based collaborative filtering approach Model-based collaborative filtering approach

Aspect model & Two-way clustering model Flexible mixture model Decouple model

Unified filtering by combining content and collaborative

filtering

What is Collaborative Filtering?

Collaborative Filtering (CF):

Making recommendation decisions for a specific user based on the judgments of users with similar tastes

Content-Based Filtering: Recommend by analyzing the content information Collaborative Filtering: Make recommendation by judgments of similar users

What is Collaborative Filtering?

Collaborative Filtering (CF): Making recommendation decisions for a specific user based on the judgments of users with similar tastes

2 3 5 1 4 Train_User 2 4 3 ? 1 Test User 4 3 3 5 1 Train_User 1

What is Collaborative Filtering?

Collaborative Filtering (CF): Making recommendation decisions for a specific user based on the judgments of users with similar tastes

2 3 5 1 4 Train_User 2 4 3

5

1 Test User 4 3 3 5 1 Train_User 1

Why Collaborative Filtering?

Advantages of Collaborative Filtering

Collaborative Filtering does not need content information as required by CBF The contents of items belong to the third-party (not accessible

  • r available)

The contents of items are difficult to index or analyze (e.g., multimedia information)

Problems of Collaborative Filtering

Privacy issues, how to share one’s interest without disclosing too much detailed information?

slide-2
SLIDE 2

Why Collaborative Filtering?

Applications Collaborative Filtering

E-Commerce Email ranking: borrow email ranking from your office mates (be careful…) Web search? (e.g., local search)

Formal Framework for Collaborative Filtering

Test User Ut

2 3

What we have:

  • Assume there are some

ratings by training users

  • Test user provides some

amount of additional training data What we do:

  • Predict test user’s rating

based training information

Rut (Oj)=

Training Users: Un

O1 O2 O3 ……Oj………… OM U1 U2 UN 4 1 1 5 2 2 Ui Objects: Om 3 2 4

Memory-Based Approaches

Memory-Based Approaches

Given a specific user u, find a set of similar users Predict u’s rating based on ratings of similar users

Issues

How to determine the similarity between users? How to combine the ratings from similar users to make the predictions (how to weight different users)?

Memory-Based Approaches

∑ ∑ ∑

− − − − =

2 _ 2 _ _ _ ,

) ) ( ( ) ) ( ( ) ) ( )( ) ( (

u u u u u u u u u u

R

  • R

R

  • R

R

  • R

R

  • R

w

t t t t t

Pearson Correlation Coefficient Similarity Average Ratings Vector Space Similarity

∑ ∑ ∑

=

2 2 ,

) ( ) ( ) ( ) (

  • R
  • R
  • R
  • R

w

u u u u u u

t t t

_ _ , ^ ,

( ( ) ) ( )

t t t t

u u u u u u u u u u

w R o R R

  • R

w − = + ∑

∑

Prediction:

How to determine the similarity between users? Measure the similarity in rating patterns between different users

Memory-Based Approaches

_ _ , ^ ,

( ( ) ) ( )

t t t t

u u u u u u u u u u

w R o R R

  • R

w − = + ∑

∑

Prediction:

How to combine the ratings from similar users for predicting? Weight similar users by their similarity with a specific user; use these weights to combine their ratings.

Memory-Based Approaches

Remove User-specific Rating Bias

2 3 5 1 4 Train_User 2 4 2 3 ? 1 Test User 4 3 3 5 1 Train_User 1

slide-3
SLIDE 3

Memory-Based Approaches

4 3 ? 1 Test User

  • 1

2

  • 2

1 Sub Mean (Train2) 0.8

  • 0.2
  • 0.2

1.8

  • 2.2

Sub Mean (Train1) 2 3 5 1 4 Train_User 2 1.33 0.333

  • 1.667

Sub Mean (Test) 4 3 3 5 1 Train_User 1

Normalize Rating

Memory-Based Approaches

4 3 ? 1 Test User

  • 1

2

  • 2

1 Sub Mean (Train2) 0.8

  • 0.2
  • 0.2

1.8

  • 2.2

Sub Mean (Train1) 2 3 5 1 4 Train_User 2 1.33 0.333

  • 1.667

Sub Mean (Test) 4 3 3 5 1 Train_User 1

Calculate Similarity: Wtrn1_test=0.92; Wtrn2_test=-0.44;

Memory-Based Approaches

4 3 ? 1 Test User

  • 1

2

  • 2

1 Sub Mean (Train2) 0.8

  • 0.2
  • 0.2

1.8

  • 2.2

Sub Mean (Train1) 2 3 5 1 4 Train_User 2 1.33 0.333

  • 1.667

Sub Mean (Test) 4 3 3 5 1 Train_User 1

Make Prediction:

Memory-Based Approaches

4 3

5

1 Test User

  • 1

2

  • 2

1 Sub Mean (Train2) 0.8

  • 0.2
  • 0.2

1.8

  • 2.2

Sub Mean (Train1) 2 3 5 1 4 Train_User 2 1.33 0.333

  • 1.667

Sub Mean (Test) 4 3 3 5 1 Train_User 1

Make Prediction:

Memory-Based Approaches

Problems with memory-based approaches

Associated a large amount of computation online costs (have to go over all users, any fast indexing approach?) Heuristic method to calculate user similarity and make user rating prediction

Possible Solution

Cluster users/items in offline manner, save for online computation cost Proposal more solid probabilistic modeling method Model-Based Approaches:

Aspect Model (Hofmann et al., 1999) – Model individual ratings as convex combination of preference factors

∑

∈

=

Z z l l l l l l

z r P z u P z

  • P

z P r u

  • P

) | ( ) | ( ) | ( ) ( ) , , (

) ( ) ( ) ( ) ( ) ( ) (

Two-Sided Clustering Model (Hofmann et al., 1999) – Assume each user and item belong to one user and item group.

vu u y u v v x l l l l l

C J I u P

  • P

r u

  • P

l l ) ( ) (

, ) ( ) ( ) ( ) ( ) (

) ( ) ( ) , , (

∑

= Ix(l)v,Jy(l)u:Indicator Variables Cvu: Associaion Parameter P(o|Z) P(Z) P(u|Z) P(r|Z) Ol Rl Ul L Z

Collaborative Filtering

slide-4
SLIDE 4

Thoughts: Previous algorithms all cluster users and objects either implicitly (memory-based) or explicitly (model-based)

– Aspect model allows users and objects to belong to different classes, but cluster them together – Two-sided clustering model clusters users and objects separately, but only allows them to belong to one single class

Collaborative Filtering Previous Work: Thoughts

Cluster users and objects separately AND allow them to belong to different classes Flexible Mixture Model (FMM)

Flexible Mixture Model (FMM):

Cluster users and objects separately AND allow them to belong to different classes

∑

=

u

  • Z

Z u

  • l

u l

  • l

u

  • l

l l

Z Z r P Z u P Z

  • P

Z P Z P r u

  • P

, ) ( ) ( ) ( ) ( ) ( ) (

) , | ( ) | ( ) | ( ) ( ) ( ) , , (

  • Training Procedure:

Annealed Expectation Maximization (AEM) algorithm E-Step: Calculate Posterior Probabilities

∑

=

u

  • Z

Z b u

  • l

u l

  • l

u

  • b

u

  • l

u l

  • l

u

  • l

l l u

  • Z

Z r P Z u P Z

  • P

Z P Z P Z Z r P Z u P Z

  • P

Z P Z P r u

  • z

z P

, ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (

)) , | ( ) | ( ) | ( ) ( ) ( ( )) , | ( ) | ( ) | ( ) ( ) ( ( ) , , | , (

P(u|Zu)

Zo Zu

P(o|Zo) P(Zo) P(Zu) P(r|Zo,Zu)

Ol Rl Ul L

Collaborative Filtering

Thoughts:

Previous algorithms address the problem that users with similar tastes may have different rating patterns implicitly (Normalize user rating)

Collaborative Filtering Previous Work: Thoughts

Thoughts:

Explicitly decouple users preference values out of the rating values Decoupled Model (DM) Nice Rating: 5 Mean Rating: 2 Nice Rating: 3 Mean Rating: 1

Decoupled Model (DM)

Decoupled Model (DM):

Separate preference value ] ,...., 1 [ k Z pref ∈

from rating (1 disfavor, k favor)

} 5 , 4 , 3 , 2 , 1 { ∈ r

ZPre

Joint Probability:

∑ ∑

=

R u

  • pre

Z Z Z R pre l Z

  • u

pre l R u l

  • l

u

  • l

l l

Z Z r P Z Z Z P u Z P Z u P Z

  • P

Z P Z P r u

  • P

, , ) ( ) ( ) ( ) ( ) ( ) ( ) (

)] , | ( ) , | ( )[ | ( ) | ( ) | ( ) ( ) ( ) , , ( “Preference-Based Graphical Model for Collaborative Filtering”, UAI’03 “A study of Mixture Model for Collaborative Filtering”, Journal of IR

ZR P(u|Zu) P(o|Zo) P(Zo) P(Zu) Zo Zu P(r|ZPre,ZR) Ol Rl Ul

L

slide-5
SLIDE 5

Experimental Data

1,2,3,4,5,6 1,2,3,4,5 Scale of ratings 129.6 87.7

  • Avg. # of rated items/User

1682 1000 Number of Movies 2000 500 Number of Users EachMovie MovieRating

Datasets:

MovieRating and EachMovie

Evaluation:

MAE: average absolute deviation of the predicted ratings to the actual ratings on items.

| ) ( | 1

^ ) ( ) (

) (

l

  • l

l Test

u R r L MAE

l

− =

∑

Vary Number of Training User

Test behaviors of algorithms with different amount of training data – For MovieRating 100 and 200 training users – For EachMovie 200 and 400 training users

Vary Amount of Given Information from the Test User

Test behaviors of algorithms with different amount of given information from test user – For both testbeds Vary among given 5, 10, or 20 items

Collaborative Filtering

0.75 0.77 0.79 0.81 0.83 0.85 0.87 0.89 0.75 0.77 0.79 0.81 0.83 0.85 0.87 0.89

Movie Rating, 100 Training Users Movie Rating, 200 Training Users

0.9 0.95 1 1.05 1.1 1.15 1.2 1.25 1.3 0.9 0.95 1 1.05 1.1 1.15 1.2 1.25 1.3

Each Movie, 400 Training Users Each Movie, 200 Training Users Given: 5 10 20 Given: 5 10 20 Given: 5 10 20 Given: 5 10 20 M A E M A E M A E M A E

Results of Flexible Mixture Model Combine Collaborative Filtering and Content-Based Filtering

Content-Based Filtering (CBF): Recommend by analyzing the content information A group of aliens visit earth…………………….. kind of friendship in which E.T learns…………

Science Fiction? Yes

Young Harry is in love and wants to marry an actress, much to the displeasure of his family….

No

Unified Filtering (UF): Combining both the content-based information and the collaborative rating information for more accurate recommendation Content information is very useful when few users have rated an object.

Unified Filtering by flexible mixture model and exponential model

Unified Filtering with mixture model and exponential model (UFME):

Mixture model for rating information: Exponential model for content information

P(Zu|U) P(o) P(u) Zo Zu R P(r|Zx,Zy) dolj Ol Ul

OlJ

∑

=

u

  • Z

Z u

  • l

l l u

  • l
  • l

l l

Z Z r P u P

  • P

u Z P d Z P r u

  • P

, ) ( ) ( ) ( ) ( ) ( ) (

) , | ( ) ( ) ( ) | ( ) | ( ) , , (

Experiment Results

  • Table. MAE results for four filtering algorithms on EachMovie
  • testbed. Four algorithms are pure content-based filtering (CBF), pure

collaborative filtering (CF), unified filtering by combining mixture model and exponential model (UFME)

1.19 1.21 1.23 1.43 CBF 1.12 1.13 1.14 1.21 CF 1.19 1.24 1.21 1.43 CBF 1.06 1.07 1.10 10 Items Given 1.05 1.08 1.17 UFME 1.05 1.08 1.17 CF 100 1.09 1.11 1.19 UFME 50 20 Items Given 5 Items Given 0 Items Given Algorithms Training Users Size

slide-6
SLIDE 6

Experiment Results

forev relat rape court mate hang dress hang passion depress food younger arrest sea abandon water door finish forget broken custom inhabit mad previou forev Cluster 5 Cluster 4 Cluster 3 Cluster 2 Cluster 1

  • Table. Five most indicative words (with highest values) for 5 movie

clusters, sorted by Each column corresponds to a different movie cluster. All listed words are stemmed.

( | )

  • P Z

w

θ

Abstract

Outline

Introduction to collaborative filtering Main framework Memory-based collaborative filtering approach Model-based collaborative filtering approach

Aspect model & Two-way clustering model Flexible mixture model Decouple model

Unified filtering by combining content and collaborative

filtering