A Preference-Based Bandit Framework for Personalized Recommendation - - PowerPoint PPT Presentation

A Preference-Based Bandit Framework for Personalized Recommendation

Maryam Tavakol and Ulf Brefeld

Paderborn, Nov 8, 2016

Introduction

2

Personalized Recommendation

Preference Learning Multi-armed bandits

Recommendation

3

Recommendation

4

Preference Model

• Item i: {Shirt, Blue, Women, Cheap}
• Item k: {Polo shirt, White, Women, Expensive}

Item i ≻ Item k:

{Shirt-Polo shirt, Blue-White, Women-Women, Cheap-Expensive}

5

zik := zi − zk

Payoff Model

• Personalized model + average component

6

User 1 User 2 … User m User 1 + User 2 + … + User m

E[rt,ik|ut = uj] = β>

t zik + θ>zik

Personalized Recommendation with Qualitative Bandit

• For t = 1, …, T:
• 1. The world generates some context
• 2. The learner chooses an action
• 3. The world reacts with a reward
• Choosing the arm with the highest mean reward +

confidence interval (General case of LinUCB)

7

Unified Optimization

• Solving the objective function in dual space
• With arbitrary loss function
• Using Fenchel-Legendre conjugate

8

Squared Loss

• The problem reduces to standard quadratic
• ptimization
• Model parameters ( , ), are obtained from

9

α θ βj

max

α

− 1 2C α>α + r>α

1 2α>[ZZ> + 1 µ( X

j

φj ⌦ φ>

j ) ZZ>]α

Squared Loss

• In the contextual bandit framework:
• Mean:
• Confidence bound:

10

β>

t zik + θ>zik

c q z>

ik(Z>Z + λI)1zik

Algorithm

11

Summary

• Personalized recommendation
• Pairwise learning in bandit framework
• Optimization in dual space
• Learning algorithm for squared loss

12

Questions?

Thanks for your attention

Email: tavakol@leuphana.de