Adaptive Hierarchical Translation-based Sequential Recommendation - - PowerPoint PPT Presentation

Adaptive Hierarchical Translation-based Sequential Recommendation

Yin Zhang, Yun He, Jianling Wang, James Caverlee

Department of Computer Science and Engineering Texas A&M University, USA

How to model User Sequences?

Highly personalized

User 1 Sequence User 2 Sequence

chronological order

How to model User Sequences?

Highly personalized

User 1 Sequence User 2 Sequence

chronological order

How to model User Sequences?

Highly personalized But have general sequence patterns and item relations patterns

User 1 Sequence User 2 Sequence

chronological order

Complementary: items can go well with each together. e.g. Mac Pro and Mac Pro charger Substitute: items that are interchangeable. e.g. Mac Pro and ThinkPad

Item Relations[1]:

User behavior changes Interacted Item Relations Personal preferred item

[1] McAuley, Julian, et al. "Image-based recommendations on styles and substitutes." SIGIR, 2015.

How to model User Sequences?

Highly personalized

User 1 Sequence User 2 Sequence

chronological order

Complementary: items can go well with each together. e.g. Mac Pro and Mac Pro charger Substitute: items that are interchangeable. e.g. Mac Pro and ThinkPad

Our Goal: Improving Sequential Recommendation with Item Multi-Relations (e.g. complementary and substitute) Inside User Dynamic Sequences

[1] McAuley, Julian, et al. "Image-based recommendations on styles and substitutes." SIGIR, 2015.

But have general sequence patterns and item relations patterns

Item Relations[1]:

Input:

Complementary Complementary

Item Complementary and Substitute Relations

Substitute

… … … Output: each user next interacted items … ? … …

User 1 Sequence User 2 Sequence

… … … … … User Sequences

dynamic

Our Goal: Improving Sequential Recommendation with Item Multi-Relations (e.g. complementary and substitute) Inside User Dynamic Sequences

How to consider item relations in user sequences? Challenges

• 1. Item relation data is extremely sparse, how to make it generalized across time in user

sequence recommendation?

2019 ThinkPad 2019ThinkPad Charger 2020 ThinkPad 2020 ThinkPad Charger

Item- Level

?

Complementary

… Item Relation Data (Sparse, Stable) User Sequence (Highly personalized, Dynamic) item can update over time

Category- Level

Computer Computer Charger Computer Computer Charger

Complementary Complementary

user sequence

Category-level Item Relation (Pairs) User Sequence (Dynamic change, Sequence)

How to consider item relations in user sequences? Challenges

• 1. Item relation data is extremely sparse, how to make it generalized across time in user

sequence recommendation?

• 2. How to effectively capture the item multiple relations inside user dynamic sequences?

How to consider item relations in user sequences? Challenges

• 1. Item relation data is extremely sparse, how to make it generalized across time in user

sequence recommendation?

• 2. How to effectively capture the item multiple relations inside user dynamic sequences?

Complementary Substitute

With the changes of user sequences, item relations can also dynamically change.

How to consider item relations in user sequences? Challenges

• 1. Item relation data is extremely sparse, how to make it generalized across time in user

sequence recommendation?

• 2. How to effectively capture the item multiple relations inside user dynamic sequences?

Complementary Substitute

With the changes of user sequences, item relations can also dynamically change.

How to consider item relations in user sequences? Challenges

• 1. Item relation data is extremely sparse, how to make it generalized across time in user

sequence recommendation?

• 2. How to effectively capture the item multiple relations inside user dynamic sequences?

Complementary Substitute

With the changes of user sequences, item relations can also dynamically change.

How to consider item relations in user sequences? Challenges

• 1. Item relation data is extremely sparse, how to make it generalized across time in user

sequence recommendation?

Complementary Substitute

• 2. How to effectively capture the item multiple relations inside user dynamic sequences?

With the changes of user sequences, item relations can also dynamically change.

• 3. How to model the dynamic joint influence of user personal preference and item

relations?

User behavior changes Interacted Item Relations Personal preferred item

How to consider item relations in user sequences? Challenges

• 2. How to effectively capture the item multiple relations inside user dynamic sequences?
• 1. Item relation data is extremely sparse, how to make it generalized across time in user

sequence recommendation?

Sequential Recommendation Methods

• Focus on the diverse

patterns of item orders in user sequences

• e.g. Markov Chains,

RNNs, CNNs, SASRec[1]

[1] Kang et al. "Self-attentive sequential recommendation." ICDM. IEEE, 2018. [2] He,et al. . "Translation-based recommendation." RecSys. 2017.

• Based on TransE, user

connects items to model ‘higher-order’ interactions:

• Since stays the same in

the user sequence, these models assume a user’s translation behavior is the same across time;

• e.g. TransRec[2]

user translation vector Item Embedding Item Embedding

⃗ ru

• Hierarchical Translation-

based Recommendation (HierTrans): the translation behavior (green line) can adaptively change according to both

• user preference and
• the relations of her

recent interacted items

• more flexible to capture

user complex dynamic preferences over time.

• e.g. HierTrans

Proposed Method: HierTrans

• 1. Building Hierarchical Temporal

Graph

• 2. Recommendation with

HierTrans

Building Hierarchical Temporal Graph

Node: Category Edge: Item relation Node: Item Edge: Each user sequence Item Multi-Relations Graph

We extend the item-level relations to category-level. captures item category-level semantic relation information. E,g, if item i complements/substitutes item j, then the category of item i is close to category of item j in complements/substitutes relation.

• Construction Method: if item i complements/substitutes

item j, then the category of item i is connected to the category of item j by complements/substitutes relation. Details refer paper and [1];

• Dense and General for dynamic changes;

Dynamic User Interactions Graph

Items are connected based on user dynamic sequences. The node connections is dynamic changed with user sequences changes.

• Construction Method: In user u sequence, if item j is next to

item i, then we connect i to j by user u.

• Graph structure benefits learning of different

patterns in the user sequences;

[1] Wang, Zihan, et al. "A path-constrained framework for discriminating substitutable and complementary products in e-commerce." WSDM. 2018.

GC GI

GC

Building Hierarchical Temporal Graph

: belong to It aggregates the influence from both item multi-relations and user sequences.

• Construction Method: if item node i in G^i

belongs to category node c in G^c, then connect node i to node c by r_b;

• Given a user sequence, by referring to r_b,

we can easily capture each item category- level relations inside user sequences;

rb

Recommendation with HierTrans

user u previous T interacted items Captures both user personal dynamic preferred patterns and item relations based on recent interacted T items. User translation vector can adaptively change based on both user’s previous interacted T items and item different relations.

GI GC

attention mechanisms

Experiments

• How well does HierTrans perform for sequential recommendation?
• What impact do the design choices of HierTrans have, e.g.

category-level relations and adaptive translation vector?

Dataset*: Three categories in Amazon (Electronics (Elec), Cell

Phones & Accessories (C & A), Home & Kitchen (H&K))

* McAuley, Julian, et al. "Image-based recommendations on styles and substitutes." SIGIR, 2015.

• BPR: Bayesian personalized ranking;
• TransE:
• TransFM:
• TransRec:
• GRU4Rec: RNN based;
• Caser: CNN based;
• SASRec: Self-attention based;

Experimental Setup: Baselines

Metrics: Following with previous sequential recommendation, we use Recall, NDCG. Translation-based Methods

Experiments: Recommendation Effectiveness

• HierTrans consistently outperforms state-of-the-art methods in recall@K and

NDCG@K;

Experiments: Impact of Influence Factors

Without adaptive translation and multiple previous heads HierTrans

• HierTrans consistently outperforms all its variations in recall@K and NDCG@K;
• The adaptive translation and category-level relations can help improve the sequential

recommendation;

Item-level relations

Conclusions and Future Work

• Category-level relations can provide denser and more general relations

across time, which can help sequential recommendation;

• We build a hierarchical temporal graph that contains item multi-relations

at the category-level and user dynamic sequences at the item-level. The hierarchical graph structure enables us to more easily extract the high-

• rder complex relation patterns among items that are revealed inside user

dynamic sequences;

• Users preference towards items dynamic changes based on both user

preference and item dynamic relations in user sequences. The adaptive translation proposed in HierTrans can effectively capture the complicated dynamic joint influence.

• Exploring the dynamic influence of different types of item relations (e.g., movies

with the same director) on user sequential behaviors;

• Future work:

Adaptive Hierarchical Translation-based Sequential Recommendation

Yin Zhang, Yun He, Jianling Wang, James Caverlee

Department of Computer Science and Engineering Texas A&M University, USA

Thank you !

Adaptive Translation Vector

Global Translation User Personal Preference Translation Item Relation Embedding

• 1. Construct the candidates of user translation choice:
• 2. Use the attention mechanisms to capture the relation

that the user would choose for the next item : In HireTrans, the translation behaviors are adaptively changed according to user preference interacted items and their relation patterns, her personal preference and her preferred item relations.