Recommendation Systems Recommendation systems aim to identify items - - PowerPoint PPT Presentation

M A T T E O B R U C A T O U N I V E R S I T Y O F M A S S A C H U S E T T S , A M H E R S T , U S A A Z Z A A B O U Z I E D N E W Y O R K U N I V E R S I T Y , A B U D H A B I , U A E A L E X A N D R A M E L I O U U N I V E R S I T Y O F M A S S A C H U S E T T S , A M H E R S T , U S A P R E S E N T E D B Y :

MATTEO BRUCATO

m a t t e o @ c s . u m a s s . e d u

Improving Package Recommendations through Query Relaxation

Recommendation Systems

— Recommendation systems aim to identify items of

interest to users

Recommended to me by Amazon before traveling to Hangzhou: 1

Improving Package Recommendations through Query Relaxation

“Package” Recommendations

— But sometimes items are actually bundled together

in packages of items

Example 1 — Amazon bundles 2

Improving Package Recommendations through Query Relaxation

“Package” Recommendations

— But sometimes items are actually bundled together

in packages of items

Example 2 — A flight package: *

* Recommended by Google Flights

3

Improving Package Recommendations through Query Relaxation

“Package” Recommendations

— But sometimes items are actually bundled together

in packages of items

Example 3 — A meal plan: 4

Improving Package Recommendations through Query Relaxation

A “Package” Query

— All recipes should have less than 25 g of fat — The entire meal plan should have:

¡ At least 1700 kcal in total ¡ Between 3 and 5 meals per day

— The meal plan should minimize the total preparation time

5

Improving Package Recommendations through Query Relaxation

Many Feasible Solutions…

6

Improving Package Recommendations through Query Relaxation

Too Many Feasible Solutions…

1704 feasible meal plans, with only 15 recipes

7

Improving Package Recommendations through Query Relaxation

A New “Big Data” Challenge

— Usually we talk about:

¡ Lots of data ¡ Lots of features

— But what about:

¡ More combinations!

— Practical challenges of “more combinations”:

¡ Computational complexity ¡

8

Improving Package Recommendations through Query Relaxation

Usability

What Could Systems Do?

9

Improving Package Recommendations through Query Relaxation

Query

• ≥ 1700 kcal in total
• Minimal prep. time

The dietitian might be willing to accept lower calories for lower preparation time

Top-1 meal plan 1,710 kcal 3 hrs 5 min

50 50 571 571 298 298 417 417 424 424 30 30 25 25 80 80

What Could Systems Do?

10

Improving Package Recommendations through Query Relaxation

Top-1 meal plan 1,710 kcal 3 hrs 5 min

50 50 571 571 298 298 417 417 424 424 30 30 25 25 80 80

Out

1 hr less! Infeasible, but perhaps better than top-1

60 60 571 571 298 298 417 417 364 364 30 30 25 25 20 20

1,650 < 1,700 kcal 2 hrs 5 min

50 50 16.2 16.2 21 21 10.7 10.7

In

Our Approach

11

— We propose a new use of query relaxation: — Usually we relax when:

¡ The query does not produce any result ¡ The query does not produce enough results

— Here, we relax to:

¡ Improve upon some aspect of the query result

Improving Package Recommendations through Query Relaxation

Package Query Relaxations

Improving Package Recommendations through Query Relaxation

12

— What is a relaxation of a package query?

Package Query

• Base Constraints – Each meal:
• ≤ 25 g of fat in each meal
• Global Constraints – The meal plan:
• ≥ 1700 kcal in total
• Cardinality Constraints:
• 3 to 5 meals
• Minimal preparation time

Constraints Objective

Coarse-grained Relaxation Fine-grained Relaxation

30

Criteria for Relaxations

Improving Package Recommendations through Query Relaxation

13

— Relaxations modify the query and thus produce a

different result than the original query

— How do we pick a good relaxation?

¡ Relaxations should improve the result ÷ In some aspects specified by the query ÷ As much as possible ¡ But they may cause some error ÷ The total error should be as low as possible

Impact of Coarse Relaxations

Improving Package Recommendations through Query Relaxation

14

— How much should we relax?

101 102 103 104 105 20 40 60 80 100

% change % relaxation

Improvement Error

Relaxing only a few constraints provides the highest impact Diminishing returns

More relaxed

Review

Improving Package Recommendations through Query Relaxation

15

— Summary so far:

¡ Package recommendations ¡ Package query relaxations ¡ Relaxation trade-off

— Rest of the talk:

¡ How do users react to relaxations? ¡ Lessons and future work

user study

How do users react to relaxations?

Improving Package Recommendations through Query Relaxation

16

— Two Research Questions:

① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed?

Let’s ask the crowd!

Dataset Description

Improving Package Recommendations through Query Relaxation

17

— Dataset

¡ 7,955 (arguably) tasty recipes extracted from allrecipes.com

Task Instructions

Improving Package Recommendations through Query Relaxation

18

We automatically generated 50 unique task configurations:

We varied these 4 constraints

Cardinality Constraint

Objective

2 Base Constraints 2 Global Constraints

Task Choices

Improving Package Recommendations through Query Relaxation

19

— For each of the 50 configurations, we showed

5 different meal plans, each removing one constraint only:

• 1. ORIGINAL
• 2. CARDINALITYRELAX
• 3. BASERELAX
• 4. GLOBALRELAX
• 5. RANDOM

— We used colors to indicate constraints adherence or violation — Results were presented sorted by preparation time

No Relaxation

Removing the cardinality constraint Removing one base constraint

Removing one global constraint A random package

Task Screenshots

Improving Package Recommendations through Query Relaxation

20 Global constraint violation and amount of violation Objective is highlighted GLOBALRELAX

Task Screenshots

Improving Package Recommendations through Query Relaxation

21 CARDINALITYRELAX Objective got worse

Task Screenshots

Improving Package Recommendations through Query Relaxation

22 BASERELAX

Task Screenshots

Improving Package Recommendations through Query Relaxation

23 ORIGINAL Objective got even worse!

Collected Data

Improving Package Recommendations through Query Relaxation

24

— Run on crowdflower.com — Each configuration completed by 10 unique workers — No worker allowed to complete more than 5

configurations

— We removed obvious spammers a posteriori:

¡ Same explanations in every task ¡ Random explanations ¡ Inconsistent explanations

— Resulting in 115 unique workers and 306 unique task

instances

Evaluation

Improving Package Recommendations through Query Relaxation

25

— The ORIGINAL plan was rejected 30% of the time

① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed? ① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed? We need to provide users with alternatives!

Evaluation

Improving Package Recommendations through Query Relaxation

26

— Relaxed plans were chosen 76% of the time

① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed? ① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed? More likely to choose a relaxed plan than the original!

70% 91%

■ When ORIGINAL is recommended ■ When ORIGINAL is not recommended

Evaluation

Improving Package Recommendations through Query Relaxation

27

① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed? ① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed?

■ Overall ■ When ORIGINAL is recommended ■ When ORIGINAL is not recommended

BASERELAX GLOBALRELAX CARDINALITYRELAX

Why Relaxations?

Improving Package Recommendations through Query Relaxation

28

— Lower preparation time was often the reason:

Additional Lessons

Improving Package Recommendations through Query Relaxation

29

— Good explanations for the bias toward BASERELAX:

(The plans had to contain 4 meals)

The workers relaxed base constraints by transforming them into global constraints!

Future Work

Improving Package Recommendations through Query Relaxation

30

— What dictates user’s sensitivity toward different

kinds of constraints?

— Impact of fine-grained relaxations — Reverse relaxations

¡ Tightening the constraints

— Additional relaxation methods

¡ Including the type of relaxation workers spontaneously applied

Summary of Contributions

Improving Package Recommendations through Query Relaxation

31

— Novel application of query relaxation — Impact of coarse relaxations — User reaction to package relaxations

Thank you!

% change % relaxation Improvement Error