A Unified Contextual Bandit Framework for Long- and Short-Term - - PowerPoint PPT Presentation

A Unified Contextual Bandit Framework for Long- and Short-Term Recommendations

Maryam Tavakol and Ulf Brefeld {tavakol,brefeld}@leuphana.de

Skopje - Sep 21, 2017

Tavakol & Brefeld, Leuphana University Lüneburg

Recommendation

2/22

Tavakol & Brefeld, Leuphana University Lüneburg

Recommendation

3/22

Tavakol & Brefeld, Leuphana University Lüneburg

Personalization

4/22

Tavakol & Brefeld, Leuphana University Lüneburg

Short-Term Zeitgeist

5/22

Tavakol & Brefeld, Leuphana University Lüneburg

Proposed Approach

• Goal: Combination of long- and short-term interests
• f users in one unified model

➡ Long-term part + Short-term component ✤ s.t.: Generality in terms of optimization

• Framework: Contextual Multi-Armed Bandit (MAB)
• e.g., LinUCB

6/22

Tavakol & Brefeld, Leuphana University Lüneburg

Unified Model

E[rt,ai|uj] = θ>

i xt

| {z }

Shortterm

+ β>

j zai

| {z }

Longterm

+ bi current user

• utcome

context item features parameters of short-term model parameters of long-term model bias term

7/22

Tavakol & Brefeld, Leuphana University Lüneburg

General Optimization

• Objective function with arbitrary loss,

inf

θ1,...,θn β1,...,βm b

1 T

T

X

t=1

V (θ>

t xt + β> t zt + bt, rt) + λ

2 X

i

kθik2 + ˆ µ 2 X

j

kβjk2

Regularization

V (·, rt)

8/22

Tavakol & Brefeld, Leuphana University Lüneburg

General Optimization

• Using the Fenchel-Legendre conjugate of loss

function in the dual space:

sup

α,1>α=0

C

T

X

t=1

V ⇤(αt C , rt) 1 2α>[( X

i

δi ⌦ δ>

i ) XX> + 1

µ( X

i

φi ⌦ φ>

i ) ZZ>]α

Kernel trick

9/22

Tavakol & Brefeld, Leuphana University Lüneburg

Optimization

• Gradient-based approaches (in dual or primal)
• Calculating the gradient depends on the loss function
• Model parameters, , are obtained from
• Kernel functions applicable

(θi, βj)

α

10/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Algorithm

11/22

Tavakol & Brefeld, Leuphana University Lüneburg

Instantiation: Squared Loss

• Conjugate of the loss function:
• Becomes a standard quadratic optimization with a

constraint

• Confidence bound: c

q x>

t (X>X)1xt + z> t (Z>Z)1zt

V ∗(−αt C , rt) = 1 2C2 α2

t − 1

C αtrt

12/22

Tavakol & Brefeld, Leuphana University Lüneburg

Instantiation: Logistic Loss

• Conjugate of the loss function:
• Confidence bound:

V ∗(−αt rt , rt) = (1 − αt Crt ) log(1 − αt Crt ) + αt Crt log( αt Crt )

c q x>

t (X>VaX)1xt + z> t (Z>VuZ)1zt

Diagonal matrix of sigmoid model

13/22

Tavakol & Brefeld, Leuphana University Lüneburg

Model Simplification

• Focus on the item model
• Short-Term:
• Short-Term+Average:
• Focus on the user model
• Long-Term:
• Long-Term+Average:

E[rt,ai] = θ>

i xt

E[rt,ai] = θ>

i xt + β>zai

E[rt,ai|uj] = β>

j zai

E[rt,ai|uj] = β>

j zai + θ>zai 14/22

Tavakol & Brefeld, Leuphana University Lüneburg

Empirical Study

• Using squared loss function
• Dataset: User transactions from Zalando*
• Baseline: Matrix Factorization (MF)
• Performance measure: normalized average rank

*www.zalando.com 15/22

Tavakol & Brefeld, Leuphana University Lüneburg

No New User/Item

• The combined approach outperforms either short- or

long-term models —but not the baseline!

16/22

Tavakol & Brefeld, Leuphana University Lüneburg

Cold Start Scenarios

• Robustness of combined model in case of new user/item

generalizes well for both cases

17/22

Tavakol & Brefeld, Leuphana University Lüneburg

Time Complexity

• The optimization time in combined model is exponential

18/22

Tavakol & Brefeld, Leuphana University Lüneburg

Short-Term Models

• The average term compensates for the new items

19/22

Tavakol & Brefeld, Leuphana University Lüneburg

Long-Term Models

• The average term compensates for the new users

20/22

Tavakol & Brefeld, Leuphana University Lüneburg

Conclusion

• The short- and long-term interests of users are

combined in one model

• Free choice of loss function and model complexity
• There is not one best model: the choice depends
• n the application

21/22

Tavakol & Brefeld, Leuphana University Lüneburg

Questions?

Thanks for your attention A Unified Contextual Bandit Framework for Long- and Short-Term Recommendations

Maryam Tavakol & Ulf Brefeld {tavakol,brefeld}@leuphana.de

Source code available at https://github.com/marytavakol/Bandits

22/22