New York University, AI Lab - - PowerPoint PPT Presentation

Александр Тужилин

New York University, Сбербанк AI Lab

Современные подходы к проблемам в рекомендательных системах

2

Outline of the Talk

• Traditional matrix completion paradigm of

recommender systems

• Going beyond the traditional matrix completion

paradigm

• Example of a non-traditional approach

– recommending products with the most valuable aspects based on customer reviews [KDD 2017]

3

Recommender Systems (RS)

4

The Traditional Paradigm

• f Recommender Systems
• Matrix R of known ratings

– rij: rating user ci assigns to item sj

• Matrix X of user attributes

– xij: attribute xj of user ci

• Matrix Y of item attributes

– yij: attribute yj of item si

• A matrix completion problem:

estimate unknown ratings in 𝑆 Solutions [AT05]:

• Collaborative Filtering (CF)
• Content-Based
• Hybrid

Example: Netflix Prize Competition

s1 s2 … sN c1 c2 … cM R

ˆ

s1 s2 … sN c1 c2 … cM R x1 x2 … xP c1 c2 … cM X y1 y2 … yQ s1 s2 … sN Y

) , , ( ˆ Y X R f R 

5

Netflix Competition Recommendation Matrix

K-PAX Life of Brian Memento Notorious U1 4 3 2 4 U2 4 5 5 U3 2 2 4 U4 3 5 2

• The Users × Items Matrix of Ratings (w/ timestamps)
• Key issue: accurate estimation of unknown ratings (RMSE)

6

Characteristics of the Traditional (Matrix Completion) Paradigm

• Two-dimensional (2D) paradigm: Users and Items

– 3 matrices: R, X and Y

• Utility of an item to a user revealed by a single rating

– binary or multi-scaled

• Recommendations of individual items provided to individual users

– Many solution via estimation of unknown ratings

7

Going Beyond the Traditional Matrix Completion Paradigm

Remove limiting assumptions of the traditional paradigm:

• Go beyond the characteristics of (a) User × Item matrix, (b) single ratings,

(c) recommending individual items to individual users [JRTZ16] “Recommender Systems—Beyond Matrix Completion” time performance Traditional New

8

Going Beyond the Traditional Matrix Completion Paradigm

• Context-Aware RSes (CARS)

– Including spatiotemporal and mobile RSes

• Multi-criteria ratings
• Aggregate ratings and recommendations to groups
• Flexible and constraint-based recommendations
• Non-rating-based approaches (e.g., ranking-based)
• New performance measures: novelty, serendipity, …
• Social RSes
• User interactions/feedback, e.g., “conversational” RSes
• Trust and privacy
• Manipulation-resistant RSes
• Additional data sources, such as customer reviews

9

Recommending Products with Valuable Aspects Based on Customer Reviews

Joint work with K. Bauman (Temple) and B. Liu (UIC)

10

Research Idea

Idea: Recommending not only a product but also the most important (positive or negative) aspects that can enhance customer experience with the product. Examples: Positive: visit “Aquagrill” and order “FISH” there. Negative: visit “Cafe X” but do not order “DESSERT” there. Aspects come from customer reviews (e.g. Yelp).

11

Recommending Products and Aspects

Try Goat Cheese Try Sweet Mango Sauce Try Lamb Curry

• r Chicken Masala

Avoid Thai Tea

12

Importance of such Recommendations Why is it important? Novel approach to RSes that provides more tangible recommendations that enhance customer experience with the products.

13

Method of Recommending the Most Valuable Aspects Input: set of historical reviews with ratings. Output: product recommendations with the most valuable aspects enhancing customer experiences.

14

Method of Recommending the Most Valuable Aspects: An Overview

1. Extracting aspects from the reviews 2. Training Sentiment Utility Logistic Model (SULM)

• aspect sentiments
• verall satisfaction

3. Calculating aspect impact on rating 4. Recommending products and aspects

15

• 1. Extracting Aspects

and Sentiments from the Reviews

• Determine set A of aspects in an application
• Aspects are characterized by a set of terms,

e.g. MEAT - {meat, pork, bbq, lamb, ..}

• For each review r determine set of aspects Ar discussed

in r and corresponding set of sentiments

• Use Opinion Parser [Liu, 2010].

16

• 1. Extracting Aspects

and Sentiments from the Reviews

Example: “(1) Had lunch in Taqueria today. (2) Ordered the taco with rice and beans and it was great. (3) The service was quick. (4) The atmosphere was dark and soothing.”

• FOOD – positive
• SERVICE – positive
• ATMOSPHERE – positive

17

• 2. Sentiment Utility Logistic Model (SULM)
• Main purpose is to estimate the overall

customer experience

• Simultaneously fits ratings and sentiments
• Identifies relative importance of the aspects in

the potential customer experience

18

• 2. (a) Training SULM: Aspect Sentiments

Expressed sentiment (OP output): Sentiment utility

• level of satisfaction
• f user with aspect of item , latent variable.

Use Matrix Factorization (Koren et al. 2009) to estimate:

19

• 2. (b) Training SULM: Explicit Sentiment
• vs. Actual Value of an Aspect

Maximize log-likelihood: Logistic function: Estimation of explicit sentiment:

Estimate 𝜄 s so that estimated values of sentiments fit the real binary sentiments ok

u i extracted by OP

0,5 1

• 6
• 4
• 2

2 4 6

20

• 2. (c) Training SULM: Overall Satisfaction

Overall level of satisfaction (latent): Overall rating (binary):

21

• 2. (d) Training SULM: Explicit Rating
• vs. Actual Value of an Experience

Binary rating estimation: Maximize log-likelihood:

0,5 1

• 6
• 4
• 2

2 4 6

22

• 2. Scheme of SULM

Estimate parameters of SULM such that 1) 2)

0,5 1

• 6
• 4
• 2

2 4 6 0,5 1

• 6
• 4
• 2

2 4 6

23

• 2. Training SULM

Simultaneously fits ratings and sentiments: Use Stochastic Gradient Descent to fit the model.

24

• 3. Aspect Impact on Rating

For a new potential review compute the impact of each aspect

• n the predicted rating as the corresponding summand from

the rating prediction part of the model

25

• 4. Recommending Products and Aspects
• identify group of aspects over which the customer has control

(e.g. Dish vs. Decor); same for the management

• identify the most valuable aspects of the potential experience
• recommend a product and its corresponding suggestions to

experience (positive) or do not experience (negative) a particular aspect

26

• 4. Recommending Aspects to Managers

Provide complimentary drink Do not talk too much during the procedure

Manager of a Spa Salon

Recommend foot massage

27

Experimental Settings Restaurants Hotels Beauty&Spa Initial 1,344,405 96,384 104,199 Filtered 602,112 5,669 5,065 Users 23,209 352 349

28

Examples of Aspects (Restaurants)

Meat Fish Dessert Money Service Decor

beef cod tiramisu price bartender design meat salmon cheesecake dollars waiter ceiling bbq catfish chocolate cost service décor ribs tuna dessert budget hostess lounge veal shark ice cream charge manager window pork fish macaroons check staff space

29

Restaurant Hotel Beauty & Spa

Total number

65 42 45

Customer can control

49 14 17

Management can control

54 29 31 Numbers of Aspects

30

Performance Measures

1. Recommendations of Aspects: how much the average rating is changed for those customers who followed the recommendations of aspects 2. Rating prediction: AUC, Precision@Top3 3. Aspect ranking: Precision@Top3, Precision@Top5

31

Baselines

• Recommending the most popular aspect of a product
• Recommending highly rated aspect of a product
• Hidden Factors as Topics (HFT) (McAuley and Leskovec 2013) - state-of-

the-art rating prediction method based on customer reviews

• Learning to Rank User Preferences Based on Phrase-Level Sentiment

Analysis across Multiple Categories (LRPPM) (Chen et al. 2016) - the latest method for predicting the list of aspects appearing in the review

32

Recommendations of Aspects (Restaurants) 65.1% 72.3% 62.9%

Average Followed Positive Not Followed Positive Conclusion: our recommendations help to get better customer experience as captured by the ratings

33

Recommendations of Aspects (Restaurants) Followed positive recommendations

Conclusion: our recommendations help to get better customer experience vs. baselines.

60,0 % 62,0 % 64,0 % 66,0 % 68,0 % 70,0 % 72,0 % 74,0 %

Customers Managers Average Most Popular Highly Rated SULM

34

Recommendations of Aspects (Restaurants) Did not follow positive recommendations

Conclusion: our recommendations help to avoid negative customer experience better than baselines

61,0 % 62,0 % 63,0 % 64,0 % 65,0 % 66,0 %

Customers Managers Average Lowest sentiment SULM

35

Recommendations of Aspects (Hotels) Followed positive recommendations

Conclusion: our recommendations help to get better customer experience vs. baselines.

50,0 % 55,0 % 60,0 % 65,0 % 70,0 %

Customers Managers Average Most Popular Highly Rated SULM

36

Recommendations of Aspects (Beauty & Spa)

67,5 % 69,0 % 70,5 % 72,0 % 73,5 % 75,0 % 76,5 %

Customers Managers Average Most Popular Highly Rated SULM

Followed positive recommendations

Conclusion: our recommendations help to get better customer experience vs. baselines.

37

Rating Prediction Performance

AUC Prescision@Top3 Application R H B&S R H B&S HFT 0.714 0.756 0.651 0.824 0.860 0.831 LRPPM 0.694 0.725 0.637 0.801 0.828 0.816 SULM 0.707 0.745 0.663 0.818 0.851 0.827 R - restaurant, H - hotel, B&S - beauty&spa

Conclusion: our rating prediction performance is comparable to the baseline performances.

38

Aspect Ranking Performance

Precision@Top3 Prescision@Top5 Application R H B&S R H B&S LRPPM 0.20 0.41 0.24 0.16 0.34 0.22 SULM 0.19 0.40 0.22 0.16 0.33 0.19 R - restaurant, H - hotel, B&S - beauty&spa

Conclusion: our aspect ranking performance is comparable to the baseline performance.

39

Conclusion

• Proposed a new method (SULM) of recommending not only

products but also the most valuable aspects enhancing customer experiences.

• Tested on 3 applications (restaurant, hotel and beauty & spas)

and showed that our recommendations lead to better customer experience as captured by the ratings.

• The proposed method

– Provides more detailed recommendations – Helps customers make more informed decisions – Helps customers get better experiences with products.

40

Future Work on Recommender Systems

• In search of the next paradigm of RSes
• My vision: current methods will be enhanced by

novel approaches from – economics – psychology

THANK YOU!

Alex Tuzhilin atuzhilin@gmail.com